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Summary 

The Program for International Student Assessment (PISA) is a system of international 
assessments that measures 15-year-olds’ capabilities in reading literacy, mathematics liter- 
acy, and science literacy every 3 years. PISA was first implemented in 2000 and is carried 
out by the Organization for Economic Cooperation and Development (OECD), an intergov- 
ernmental organization of industrialized countries. Each PISA data-collection effort 
assesses one subject area in depth, even as all three are assessed in each cycle so that par- 
ticipating countries have an ongoing source of achievement data in every subject area (fig- 
ure 1). In addition to the major subject areas of reading literacy, mathematics literacy, and 
science literacy, PISA also measures general or cross-curricular competencies such as 
learning strategies. In this second cycle, PISA 2003, mathematics literacy was the subject 
area assessed in depth, along with the new cross-curricular area of problem solving. Major 
findings for 2003 in mathematics literacy and problem solving are provided here, as well as 
brief discussions of student performance in reading literacy and science literacy and 
changes in performance between 2000 and 2003. 

U.S. Performance in Mathematics Literacy and Problem Solving 

In 2003, U.S. performance in mathematics literacy and problem solving was lower than the 
average performance for most OECD countries (tables 2 and 3). The United States also 
performed below the OECD average on each mathematics literacy subscale representing a 
specific content area ( space and shape, change and relationships, quantity, and uncertainty). 
This is somewhat different from the PISA 2000 results, when reading literacy was the 
major subject area, which showed the United States performing at the OECD average 
(Lemke et al. 2001). 

Along with scale scores, PISA 2003 also uses six proficiency levels (levels 1 through 6, with 
level 6 being the highest level of proficiency) to describe student performance in mathemat- 
ics literacy (exhibit 5) and three proficiency levels (levels 1 through 3, with level 3 being the 
highest level of proficiency) to describe student performance in problem solving (exhibit 9). 

In mathematics literacy, the United States had greater percentages of students below level 1 
and at levels 1 and 2 than the OECD average percentages (figure 5, table B-6). The United 
States also had a lower percentage of students at levels 4, 5, and 6 than the OECD average 
percentages. Results for each of the four mathematics content areas followed a similar pat- 
tern. In problem solving, the United States also had greater percentages of students below 
level 1 and at level 1 than the OECD average percentages, and a lower percentage of students 
at levels 2 and 3 than the OECD average percentages (figure 8, table B-15). 

This is also somewhat different from the PISA 2000 reading literacy results, which showed 
that while the percentages of U.S. students performing at level 1 and below were not meas- 
urably different from the OECD averages, the United States had a greater percentage of 
students performing at the highest level (level 5) compared to the OECD average (Lemke et 
al. 2001). In mathematics literacy and problem solving in 2003, even the highest U.S. achiev- 
ers (those in the top 10 percent in the United States) were outperformed on average by their 
OECD counterparts (figures 4 and 7, tables B-4 and B-13). 

There were no measurable changes in the U.S. scores from 2000 to 2003 on either the space 
and shape subscale or the change and relationships subscale, the only content areas for 
which trend data from 2000 to 2003 are available (table B-11). In both 2000 and 2003, about 
two-thirds of the other participating OECD countries outperformed the United States in 
these content areas. 
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U.S. Performance in Reading Literacy and Science Literacy 

The U.S. average score in reading literacy was not measurably different from the OECD 
average in 2000 or 2003 (figure 9, table B-16), nor was there any measurable change in the 
U.S. reading literacy score from 2000 to 2003. 

The U.S. score was below the OECD average science literacy score in 2003 (figure 9, table B- 
17). There was no measurable change in the U.S. science literacy score from 2000 to 2003. 

Differences in Performance by Selected Student Characteristics 

Sex 

Males outperformed females in mathematics literacy in the United States and in two-thirds 
of the other countries (figure 10, table B-18). Within the United States, greater percentages 
of male students performed at level 6 (the highest level) than female students in mathemat- 
ics literacy, but larger percentages of females were not seen at lower levels (below level 1 
and levels 1 through 5; table B-19). In other words, differences in the overall scores between 
males and females in the United States were due at least in part to the fact that a higher 
percentage of males were found among the highest performers, not to a higher percentage of 
females found among the lowest performers. 

In the majority of the PISA 2003 countries (32 out of 39 countries), including the United 
States, there were no measurable differences in problem-solving scores by sex (figure 10, 
table B-21). However, females outscored their male peers in problem solving in six of the 
seven remaining participating countries, as well as at the OECD average. Males outscored 
females in problem solving in Macao-China. 

Socioeconomic Background 

In 2003, a few countries showed stronger relationships between socioeconomic background 
(as measured by parental occupational status) and student performance than the United 
States, while more showed weaker relationships. In 2003, the relationship between socio- 
economic background and student performance in mathematics literacy was stronger in 5 
countries (Belgium, Czech Republic, Germany, Hungary, and Poland) than in the United 
States, while 11 countries had weaker relationships (table B-25). Three of the same five 
countries (Belgium, Germany, and Hungary) had stronger relationships between socioeco- 
nomic background and problem-solving performance than the United States, while 12 had 
weaker relationships. 

Race/Ethnicity 

In the United States in PISA 2003, Blacks and Hispanics scored lower on average than 
Whites, Asians, and students of more than one race in mathematics literacy and problem 
solving (figure 11, table B-26). Hispanic students, in turn, outscored Black students. In 
both mathematics literacy and problem solving, the average scores for Blacks and Hispanics 
were below the OECD average scores, while scores for Whites were above the OECD 
average scores. 

For further results from PISA 2003, see the Organization for Economic Cooperation and 
Development (OECD) publication Learning for Tomorrow’s World — First Results From PISA 
2003 , available at http://www.pisa.oecd.org (OECD 2004). A technical report for PISA 2003 — 
which describes in detail all the procedures used in the design, data collection, quality con- 
trol, and analysis for the study, as well as the PISA 2003 data itself — will also be made avail- 
able at that site. 
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Introduction 

PISA in Brief 

The Program for International Student 
Assessment (PISA) is a system of interna- 
tional assessments that measures 15-year- 
olds’ capabilities in reading literacy, mathe- 
matics literacy, and science literacy every 3 
years. PISA was first implemented in 2000 
(figure 1). 

PISA is sponsored by the Organization for 
Economic Cooperation and Development 
(OECD), an intergovernmental organization 
of 30 industrialized nations. In 2003, 41 coun- 
tries participated in PISA, including 30 
OECD countries and 11 non-OECD coun- 
tries (table 1). Of those 41 countries, com- 
parisons for 39 countries (29 OECD coun- 
tries and 10 non-OECD countries) are pro- 
vided in this report. Data for one country, 
Brazil, were not available at the time of 
report production, and data for one other, the 
United Kingdom, are not discussed due to 
low response rates. 



Figure 1. Program for International Student 
Assessment (PISA) cycle 




2006 

( 2015 ...) 



NOTE:The subject in all capital letters in each assessment 
cycle is the major domain for that cycle. 

SOURCE: Organization for Economic Cooperation and 
Development (OECD), Program for International Student 
Assessment (PISA), 2003. 



Table 1. Participation in the Program for 

International Student Assessment 
(PISA), by country: 2000 and 2003 



Country 


2000 


2003 


Organization for Economic 
Cooperation and Development 
(OECD) countries 


Australia 


• 


• 


Austria 


• 


• 


Belgium 


• 


• 


Canada 


• 


• 


Czech Republic 


• 


• 


Denmark 


• 


• 


Finland 


• 


• 


France 


• 


• 


Germany 


• 


• 


Greece 


• 


• 


Hungary 


• 


• 


Iceland 


• 


• 


Ireland 


• 


• 


Italy 


• 


• 


Japan 


• 


• 


Korea, Republic of 


• 


• 


Luxembourg 


• 


• 


Mexico 


• 


• 


Netherlands' 


• 


• 


New Zealand 


• 


• 


Norway 


• 


• 


Poland 


• 


• 


Portugal 


• 


• 


Slovak Republic 




• 


Spain 


• 


• 


Sweden 


• 


• 


Switzerland 


• 


• 


Turkey 




• 


United Kingdom 2 


• 


• 


United States 


• 


• 


Non-OECD countries 


Brazil 3 


• 


• 


Hong Kong-China 




• 


Indonesia 




• 


Latvia 


• 


• 


Liechtenstein 


• 


• 


Macao-China 




• 


Russian Federation 


• 


• 


Serbia and Montenegro 




• 


Thailand 




• 


Tunisia 




• 


Uruguay 




• 



'Due to low response rates, PISA 2000 data for the 
Netherlands are not discussed in this report. For information 
on the results for the Netherlands, see OECD (2001). 

2 Due to low response rates, PISA 2003 data for the United 
Kingdom are not discussed in this report. 

3 Although Brazil participated in PISA 2003, its data were not 
available in time for production of this report. 

NOTE: A indicates that the country participated in PISA 
in the specific year. Because PISA is principally an OECD 
study, non-OECD countries are displayed separately from the 
OECD countries. 

SOURCE: Organization for Economic Cooperation and 
Development (OECD), Program for International Student 
Assessment (PISA), 2000 and 2003. 
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Each PISA data-collection effort assesses 
one subject area in depth, even as all three 
are assessed in each cycle so that partici- 
pating countries have an ongoing source of 
achievement data in every subject area. In 
addition to the reading literacy, mathematics 
literacy, and science literacy, PISA also 
measures general or cross-curricular com- 
petencies such as learning strategies. In 
this second cycle, PISA 2003, mathematics 
literacy was the subject area assessed in 
depth, along with the new cross-curricular 
area of problem solving. In 2006, PISA will 
focus on science literacy. Results from 
PISA 2000, which focused on reading litera- 
cy, are described in Lemke et al. (2001) and 
Organization for Economic Cooperation and 
Development (OECD) (2001). In addition, a 
series of thematic reports exploring topics 
related to reading literacy in greater depth 
are available through http://www.pisa.oecd.org 
(see also the PISA resources and publica- 
tions section of this report for information 
about PISA publications). 

This report focuses on the performance of 
U.S. students in the two major areas 
assessed in 2003, mathematics literacy and 
problem solving. Achievement in the minor 
domains of reading literacy and science lit- 
eracy in 2003 is also presented, and differ- 
ences in achievement by selected student 
characteristics are covered in the final sec- 
tion. 

The Unique Contribution of PISA 

The United States has conducted surveys of 
student achievement at a variety of grade 
levels and in a variety of subject areas 
through the National Assessment of 
Educational Progress (NAEP) for many 
years. NAEP provides a regular benchmark 
for states and the nation and a means to 
monitor progress in achievement over time. 



In order to provide a critical external per- 
spective on the achievement of U.S. stu- 
dents through comparisons to other nations, 
the United States participates at the inter- 
national level in PISA, the Progress in 
International Reading Literacy Study 
(PIRLS), and theTrends in International 
Mathematics and Science Study (TIMSS). 1 
TIMSS and PIRLS seek to measure stu- 
dents’ mastery of specific knowledge, skills, 
and concepts, and are designed to reflect 
curriculum frameworks in the United States 
and other participating countries. 

PISA provides a unique and complementary 
perspective to these studies by not focusing 
explicitly on curricular outcomes, but on the 
application of knowledge in reading, mathe- 
matics, and science to problems with a real- 
life context (OECD 1999). The framework for 
each assessment area is based upon con- 
tent, processes, and situations or contexts. 
For example, for mathematics literacy, the 
content is made up of major mathematical 
ideas, such as space and shape and uncer- 
tainty. The processes describe what strate- 
gies students use to solve mathematics 
problems, such as making connections or 
performing simple calculations. The situa- 
tions or contexts refer to the kinds of places 
in which students might encounter mathe- 
matical problems, such as personal or edu- 
cational. Assessment items are then devel- 
oped based on these descriptions. 

PISA uses the terminology of ‘‘literacy” in 
each subject area to denote its broad focus 
on application of knowledge and skills; that 
is, PISA seeks to ask if 15-year-olds are 
mathematically literate, or to what extent 
they can apply mathematical knowledge and 
skills to a range of different situations they 
may encounter in their lives. Literacy itself 
refers to a continuum of skills — it is not a 
condition that one has or does not have (i.e., 
literacy or illiteracy), but rather each per- 
son’s skills place them in a particular place 
on the literacy continuum. 



^he United States has also participated in international comparative assessments of civics knowledge and skills (CivEd 1999) 
and adult literacy (International Adult Literacy Survey [IALS 1994] and Adult Literacy and Lifeskills Survey [ALL 2003]). 
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The target age of 15 allows countries to 
compare outcomes of learning as students 
near the end of compulsory schooling. 
PISA’s goal is to answer the question "what 
knowledge and skills do students have at 
age 15?” taking into account schooling and 
other factors that may influence their per- 
formance. In this way, PISA’s achievement 
scores represent a "yield” of learning at age 
15, rather than a direct measure of attained 
curriculum knowledge at a particular grade 
level, since 15-year-olds in the United 
States and elsewhere come from several 
grade levels and are enrolled in a variety 
of classes (figures 2 and 3, tables B-1 and 
E3-2). 

How PISA 2003 Was Conducted 

PISA 2003 was sponsored by the OECD and 
carried out at the international level through 
a contract with the PISA Consortium, led by 
the Australian Council for Educational 
Research (ACER). 2 The National Center for 
Education Statistics (NCES) of the Institute 
of Education Sciences at the U.S. 
Department of Education was responsible 
for the implementation of PISA in the 
United States. Data collection in the United 
States was carried out through a contract 
with Westat. A review panel (see appendix 
C for a list of members) provides input on 
the development and dissemination of PISA 
(andTIMSS) in the United States. 

PISA 2003 was a 2-hour paper-and-pencil 
assessment of 15-year-olds collected from 
nationally representative samples in partici- 
pating countries. Like other large-scale 
assessments, PISA was not designed to 
provide individual student scores, but rather 
national and sub-national estimates of per- 
formance. Every student in PISA 2003 was 
assessed in mathematics literacy; reading, 
problem solving, and science questions were 
spread among students (for more informa- 
tion on PISA 2003’s design, see the techni- 
cal notes in appendix A). 



PISA 2003 was administered between March 
and May 2003. The U.S. sample included 
both public and private schools, randomly 
selected and weighted to be representative 
of the nation. 3 In the United States, to 
improve response rates (a response rate of 
approximately 50 percent was projected for 
the end of the data collection period) and 
better accommodate school schedules, a 
second testing window was opened from 
September through November 2003. In total, 
262 schools and 5,456 students participated 
in PISA 2003 in the United States. An over- 
all weighted school response rate of 65 per- 
cent before the use of replacement schools 
and a weighted student response rate of 83 
percent was achieved after testing in the 
second window was complete (see technical 
notes in appendix A for additional details on 
sampling, administration, response rates, 
and other issues). 

For further results from PISA 2003, see the 
Organization for Economic Cooperation and 
Development (OECD) publication Learning 
for Tomorrow’s World — First Results From 
PISA 2003, available at ht tp://www.pisa.oecd.org 
(OECD 2004). A technical report for PISA 
2003 — which describes in detail all the pro- 
cedures used in the design, data collection, 
quality control, and analysis for the study, as 
well as the PISA 2003 data itself — is also 
available at that site. 

This report provides results for the United 
States in relation to the other countries par- 
ticipating in PISA 2003, distinguishing 
OECD countries and non-OECD countries. 
All differences described in this report have 
been tested for statistical significance at 
the .05 level. Additional information on sta- 
tistical procedures used in this report is pro- 
vided in the technical notes. 



2 The PISA Project Consortium consists of the Australian Council for Educational Research (ACER), the Netherlands National 
Institute for Educational Measurement (CITO), EducationalTesting Service (ETS, USA), National Institute for Educational Policy 
Research (NIER, Japan), and Westat (USA). 

3 The sample frame data for the United States for public schools were from the Common Core of Data (CCD), and the data for 
private schools were from the Private School Survey (PSS). Any school containing at least one 7th- through 12th-grade class as 
of the school year 2000-01 was included on the school sampling frame. 
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Figure 2. Percentage distribution of U.S. 15-year-old students, by grade: 2003 
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NOTE: Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 



Figure 3. Percentage distribution of U.S. 15-year-old students, by type of mathematics class: 



2003 



Pre-algebra or 
Other general math 

8 9 




Algebra I 
29 



NOTE: Type of class refers to the mathematics class in which the student was enrolled at the time of assessment. Detail may 
not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 



PISA 2003 Results From the U.S. Perspective 



U.S. Performance in 
Mathematics Literacy 

PISA’s major focus in 2003 was mathematics 
literacy. Mathematics literacy is defined as: 

...an individual’s capacity to identify and 
understand the role that mathematics plays 
in the world, to make well-founded judge- 
ments and to use and engage with mathe- 
matics in ways that meet the needs of that 
individual’s life as a constructive, con- 
cerned, and reflective citizen. (OECD 2003, 
P-24) 

PISA’s emphasis is on the ability to apply a 
range of knowledge and skills to a variety of 
problems with real-life contexts. In the 
PISA 2003 mathematics literacy assess- 
ment, students completed exercises 
designed to assess their capabilities in 
using a range of mathematical competen- 
cies, grouped and described as "competen- 
cy clusters.” These clusters — reproduction, 
connections, and reflection — describe sets 
of skills students may use to solve prob- 
lems. The reproduction cluster involves the 
reproduction of the practiced material and 
performing routine operations. The connec- 
tions cluster calls for integration and con- 
nection of material, and the modest exten- 
sion of practiced material. The reflection 
cluster relates to students’ abilities in 
advanced reasoning, argumentation, 
abstraction, generalization, and modeling 
applied to new contexts. 

The problems themselves were designed to 
come from the variety of situations (person- 
al, educational/occupational, public, or sci- 
entific) that students encounter, and to have 
a real-life context. The mathematical con- 
tent of the problems was drawn from four 
overarching ideas: space and shape, change 
and relationships, quantity, and uncertainty. 



These overarching ideas represent a way to 
organize mathematical content broadly and 
encompass many traditional curricular areas 
such as algebra or geometry (see also 
Steen 1990). 

• Space and shape includes recognizing 
shapes and patterns, describing, encod- 
ing, and decoding visual information, 
understanding dynamic changes to 
shapes, understanding similarities and 
differences and relative positions, and 
understanding the relationship between 
visual representations and real shapes 
and images. 

• Change and relationships covers the rep- 
resentation of change, including mathe- 
matical functions such as linear, expo- 
nential, or logistic, as well as data analy- 
sis needed to specify relationships or 
translate between representations. 

• Quantity focuses on quantitative reason- 
ing (including number sense, estimating, 
mental arithmetic, understanding mean- 
ing of operations, having a feel for the 
magnitude of numbers, and computa- 
tions) and understanding of numerical 
patterns, counts, and measures. 

• Uncertainty includes the two related topics 
of data and chance, or statistics and prob- 
ability, including data analysis and graphic 
and numeric representations of data. 

A comparative analysis of the NAEP, PISA, 
andTIMSS mathematics assessments spon- 
sored by NCES found that the 2003 PISA 
mathematics literacy assessment used far 
fewer multiple-choice items than NAEP or 
TIMSS. PISA also had a much stronger con- 
tent focus on the "data” area (which often 
deals with using charts and graphs), which 
fits with PISA’s emphasis on using materi- 
als with a real-world context (see technical 
notes for more information on the results of 
the assessment comparisons). 4 



4 See Neidorf,T.S., Binkley, M., Gattis, K., and Nohara, D. (forthcoming) and the technical notes in appendix A for more informa- 
tion. Other comparative analyses focus on assessments of science and reading in PISA, NAEP, TIMSS, and PIRLS. See 
Neidorf,T.S., Binkley, M., and Stephens, M. (forthcoming); Binkley, M., and Kelly, D. (2003); Binkley, M., Afflerbach, R, and Kelly, D. 
(forthcoming); and Nohara, D. (2001). 
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Sample mathematics literacy items for each 
of these areas and student responses are 
shown here. For more information about the 
mathematics literacy domain, refer to The 
PISA 2003 Assessment Framework: 
Mathematics, Reading, Science, and Problem 
Solving Knowledge and Skills (OECD 2003). 
Additional mathematics literacy sample 
items can be found at ht tp://nces.ed.aov/ 
survevs/pisa . in the PISA 2003 framework 
document referenced above, in Measuring 
Student Knowledge and Skills: The PISA 2000 
Assessment of Reading, Mathematical and 
Scientific Literacy (OECD 2000) and in 
Sample Tasks from the PISA 2000 
Assessment: Reading, Mathematical and 
Scientific Literacy (OECD 2002). 
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Exhibit 1. Space and shape sample item: 2003 



Cubes 



Question?!): CUBES 

In Ihis photograph you see six. dice, labeled fa) to {!>. For all dice there is a rule: 
The total number of dots on two opposrte faces of each die is always seven. 




Write in each bo* the dumber ol dote on the bottom late Of the dice corresponding to 
the photograph. 



(a) 


fb) 


{c) 


1 


5 


+ 


2_ 




5 


(d> 


fe) 


(!) 



RESULTS: 

U.S percent correct 62.7 

OECD percent correct , . 68 0 

I tem dificulty: Level 2 
Siluation: Occupational 
Competency: Reproduction 



SOURCE: Organization for Economic Cooperation and Development (OECD), Program for 
International Student Assessment (PISA), 2003. 
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Exhibit 2. Change and relationships sample item: 2003 



The Best Car 



A car magazine uses a rating system to evaluate new cars, and gives the award ol 'The 
Car of the Year' to the car with the highest total score. Five new cars arc being 
evaluated, and their ratings are shown in the table 



Car 


Satety 

Features- 


Fuel 

EHicieney 


External 

Appearance 


Internal 

Fittings- 




(S) 


in 


IE) 


m 


Ca 


3 


i 


2 


3 


M? 


~~T~ 


2 


? 


~T~ 




3 


i 


3 


2 


Mi 


t 


3 


3 


3 


KK 


3 


2 


3 


S' 



The ratings are interpreted as follows 

3 points - Excellent 
2 points = Good 
1 poini = Fair 



Question 5: THE BEST CAR 



IUTQ4QQ1 



To calculate the lotat score lor a car, the car magazine uses the following rule, which is a 
weighted sum of the individual score points: 

Total Score = (3:kS)+F + E + T 

Calculate the ictal score ter Car "Ca". Write your answer in the space below. 



Total score for 'Ca": 



JS 



RESULTS: 




U.S. percent correct 


..74.6 


OECD percent correcl 


- 72.5 


Item difficulty: Level 2 




Situation: Public 




ConnpQlQncy: Reproduction 





SOURCE: Organization for Economic Cooperation and Development (OECD), Program for 
International Student Assessment (PISA), 2003. 
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Exhibit 2. Change and relationships sample item: 2003 — Continued 



The Best Car 



Question 6; THE BEST CAR 

The manufacturer of car "Ca" thought the rule for line total score was unfair. 

Write down a rule lor calculating the total score so that Car 'Ca" will be the winner. 

Your rule should include all four of the variables, and you should wrile down your rule by 
filling in positive numbers in Ihe four spaces m the equation below. 

3 J 

Total score x S ■+ x F +- x E +- x T. 

35 4F 4 EL 43T 

ca •= ZJ 
hnt = It* 

5p ^ 19 

Nr = 

kk - zo 



RESULTS: 




U.S. percent correct. ... , 


.... 20,2 


OECD percent correct 


25.4 


Item difficulty: Level 5 




Situation. Public 




Competency: Rofleciion 





SOURCE: Organization for Economic Cooperation and Development (OECD), Program for 
International Student Assessment (PISA), 2003. 
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Exhibit3. Quantity sample item: 2003 



Exchange Rate 



totfi-Ling Irom Singapore was preparing to go to SquM Ain-ca lor 3 months as an 
^Change student $he needed Id change Seme Singapore dollars (SOD) into South 
African rand (2 A Fit. 



Question 14: EXCHANGE RATE 

On returning to Singapore after 3 months. MeMing had 3S00ZAR left. She changed 
this back to Singapore dollars, ogling that the exchange rale had changed (g 

1 SCO = 4.0 ZAR 

How much money in Singapore dollar did Mei-Lmg get 1 ? 

915 

Answer: 



RESULTS: 




U.S. percent cor reel 


67.3 


OECD percent correct 


73.9 


Item dil l> rally: Level 2 




SiLuabon: Public 




Compatency: Reproduclicri 





SOURCE: Organization for Economic Cooperation and Development (OECD), Program for 
International Student Assessment (PISA), 2003. 
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Exhibit3. Quantity sample item: 2003 — Continued 



Exchange Rate 



AHtJOGS- Or iV fj au 



Question 15: EXCHANGE RATE 

Du-nng these 3 months the exchange rate had changed (rom> 4.2 to 4.D ZAR per SGD. 

Was it in Men- Ling’s favor that the exchange rate now was 4.0 ZAR instead of 4.2 ZAR 
when she changed her South African rand back to Singapore dollars? Give an 
explanation to support your answer. 



ijp 40, . then 3900/4 ^ 915 
ip 42- t . -then 3900/4.2- = 92-S.51 

^es Because she would receive furore 
Money 



RESULTS: 

U.S. percent correct 37 2 

OECD percent correct 40.3 

Item difficulty: Level 4 
Situation: Public 
Competency: Ftettechon 



SOURCE: Organization for Economic Cooperation and Development (OECD), Program for 
International Student Assessment (PISA), 2003. 
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Exhibit 4. Uncertainty sample item: 2003 



Test Scores 



Question 32: TEST SCORES 



US tSQQt ri ; y 



The diagram below shows the results on a Science lest for two groups. ! a cosed as Group 
A and Grou p B. 

The mean score tor Group A rs 62.0 and the mean tor Group B is 64.6. Students pass 
this test when ttietr score is SO or above. 

~ Scores on a Science lest 







0a 

O 



£F> 



-■u 



o> 



C; 

T7 



(B 

id 






GJi 

r^ 



ts 

w 



s s 



Group A 
I student failed 
I ( Pass 



Score 



Group A L Group 8 



S § 3 

Group & 

2 , students failed 

tO Pass 



Locking at the diagram, the teacher claims that Group B dip better than Group A in this 
lest. 

The students in Group A don't agree- wth their teacher, They try to convince the teacher 
lhat Group B may not necessarily have done better 

Give o-ne mathematical argument, using Ihe graph, that the students in Group A could 
use, 



<^roup A could arGue -that I f 
People -ProiVi -their Group passed, 
ar\d onjy IO ^roivt Group & did. 



RESULTS: 

u.S. percent correct 39.fi 

OECD percent correct 32.2 

Item ditticuity: Level 5 
Situation: Educational 
Competency: Connections 



SOURCE: Organization for Economic 
Cooperation and Development 
(OECD), Program for International 
Student Assessment (PISA), 2003. 
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Combined mathematics literacy scores are 
reported on a scale with a mean of 500 and 
standard deviation of 100. 5 Fifteen-year-old 
students in the United States had an aver- 
age score of 483 on the combined mathemat- 
ics literacy scale, lower than the OECD 
average score of 500 (tables 2 and B-3). U.S. 
students were less mathematically literate 
than their peers in 20 of the other 28 OECD 
countries and 3 of the 10 non-OECD coun- 
tries. Eleven countries (5 OECD countries 
and 6 non-OECD countries) reported lower 
scores compared to the United States in 
mathematics literacy. 

U.S. students also had lower scores than 
the OECD average scores for each of the 
four content area subscales ( space and 
shape, change and relationships, quantity, 
and uncertainty). Twenty-four countries (20 
OECD and 4 non-OECD countries) outper- 
formed the United States on the space and 
shape subscale, 21 countries (18 OECD and 
3 non-OECD countries) outperformed the 
United States on the change and relation- 
ships subscale, 26 countries (23 OECD and 3 
non-OECD countries) outscored the United 
States on the quantity subscale, and 19 
countries (16 OECD and 3 non-OECD coun- 
tries) outscored the United States on the 
uncertainty subscale. 



5 Because the average was set for the combined mathematics literacy scale, average scores for the mathematics literacy subscales 
differ slightly from 500. PISA 2000 mathematics literacy scores were re-scaled using the greater detail in PISA 2003 data in order to 
provide a more complete measure of achievement than that available in 2000. See technical notes in appendix A for more informa- 
tion on scaling. PISA's intent for each subject area is to draw baseline information for describing changes and trends in achieve- 
ment from the cycle in which that subject area is the major domain. The use of minor domains allows PISA to provide indicative 
information about changes in performance over time; however, changes in a subject area are best measured from the cycle in which 
it is the major domain. Thus, changes in reading literacy achievement are based upon PISA 2000 data, when reading literacy was the 
major domain, and changes in mathematics literacy scores, in turn, are based upon this 2003 cycle. Science literacy scores from 2000 
and 2003 may be re-scaled based up on the much greater detail for science literacy which will be available in 2006. 
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Table 2. Average combined mathematics literacy scores and subscale scores of 15-year-old 
students, by country: 2003 



Combined mathematics literacy 
Country Score 



OECD average 500 



Mathematics subscales 



Space and shape 



Country 



OECD average 



Change and relationships 



Score Country 



496 OECD average 



Score 



OECD countries 




Poland 


490 


Hungary 


490 


Spain 


485 


United States 


483 


Portugal 


466 


Italy 


466 


Greece 


445 


Turkey 


423 


Mexico 


385 



OECD countries 


Japan 


553 


Korea 


552 


Switzerland 


540 


Finland 


539 


Belgium 


530 


Czech Republic 


527 


Netherlands 


526 


New Zealand 


525 


Australia 


521 


Canada 


518 


Austria 


515 


Denmark 


512 


France 


508 


Slovak Republic 


505 


Iceland 


504 


Germany 


500 


Sweden 


498 


Poland 


490 


Luxembourg 


488 


Norway 


483 


Hungary 


479 


Spain 


476 


Ireland 


476 


United States 


472 


Italy 


470 


Portugal 


450 


Greece 


437 


Turkey 


417 


Mexico 


382 



OECD countries 




Slovak Republic 494 

Norway 488 

Luxembourg 487 

United States 486 

Poland 484 

Spain 481 



Portugal 


468 


Italy 


452 


Greece 


436 


Turkey 


423 


Mexico 


364 



Non-OECD countries 



Latvia 


483 


Russian Federation 


468 


Serbia and Montenegro 


437 


Uruguay 


422 


Thailand 


417 


Indonesia 


360 


Tunisia 


359 



Non-OECD countries 


Hong Kong-China 


558 


Liechtenstein 


538 


Macao-China 


528 


Latvia 


486 


Russian Federation 


474 


Serbia and Montenegro 


432 


Thailand 


424 


Uruguay 


412 


Indonesia 


361 


Tunisia 


359 



Non-OECD countries 



Latvia 


487 


Russian Federation 


477 


Serbia and Montenegro 


419 


Uruguay 


417 


Thailand 


405 


Tunisia 


337 


Indonesia 


334 



Hong Kong-China 

Liechtenstein 

Macao-China 



550 

536 

527 



Hong Kong-China 

Liechtenstein 

Macao-China 



540 

540 

519 



See notes at end of table. 
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Table 2. Average combined mathematics literacy scores and 
subscale scores of 15-year-old students, by country: 
2003 — Continued 



Mathematics subscales 



Quantity 



Uncertainty 



Score Country 



Score 



OECD average 



OECD countries 




United States 


476 


Italy 


475 


Portugal 


465 


Greece 


446 


Turkey 


413 


Mexico 


394 



OECD countries 




Austria 494 

Poland 494 

Germany 493 

Luxembourg 492 

United States 491 

Hungary 489 

Spain 489 




Non-OECD countries 



Hong Kong-China 


545 


Liechtenstein 


534 


Macao-China 


533 


Latvia 


482 


Russian Federation 


472 


Serbia and Montenegro 


456 


Uruguay 


430 


Thailand 


415 


Tunisia 


364 


Indonesia 


357 



Non-OECD countries 



Hong Kong-China 


558 


Macao-China 


532 


Liechtenstein 


523 


Latvia 


474 


Russian Federation 


436 


Serbia and Montenegro 


428 


Thailand 


423 


Uruguay 


419 


Indonesia 


385 


Tunisia 


363 



□ Average is significantly higher than the U.S. average 
ilj Average is not significantly different than the U.S. average 
■ Average is significantly lower than the U.S. average 

NOTE: Statistical comparisons between the U.S. average and the Organization for 
Economic Cooperation and Development (OECD) average take into account the contri- 
bution of the U.S. average toward the OECD average. The OECD average is the average 
of the national averages of the OECD member countries with data available. Because the 
Program for International Student Assessment (PISA) is principally an OECD study, the 
results for non-OECD countries are displayed separately from those of the OECD coun- 
tries and are not included in the OECD average. Due to low response rates, data for the 
United Kingdom are not discussed in this report. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program 
for International Student Assessment (PISA), 2003. 
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On average, the highest U.S. achievers 
(those in the top 10 percent of U.S. students) 
were outperformed by their OECD counter- 
parts (figure 4, table B-4). To be in the top 10 
percent in the United States, students had 
to score 607 or higher, while on average 
across the OECD countries, students would 
have had to score 628 or higher to be in the 
top 10 percent. Scores for the top 10 percent 
of students within countries ranged from 466 
or better in Indonesia andTunisia to 672 or 
better in Hong Kong-China. Low performers 
in the United States (those in the bottom 10 
percent) had a cutoff score of 356 or lower, 
which was lower than the cutoff score of 369 
or lower for the OECD average. There was 
approximately a 251 point score difference, 
or about two and a half standard deviations, 
between the cutoff scores for the top 10 per- 
cent and the bottom 10 percent of 15-year-old 
students for mathematics literacy in the 
United States, compared to about a 259 point 
difference using the OECD average scores. 

The standard deviation (which measures the 
spread of scores around the average) for the 
United States (95), in fact, was lower than 
the OECD average standard deviation of 100 
(table B-5). Sixteen countries (10 OECD and 
6 non-OECD countries) showed less varia- 
tion in performance than the United States, 
while three countries (Belgium, Germany, 
and Uruguay) had larger standard deviations. 



Along with scale scores, PISA 2003 also uses 
six proficiency levels (levels 1 through 6, with 
level 6 being the highest level of proficiency) 
to describe student performance in mathe- 
matics literacy (exhibit 5). An additional level 
(below level 1) encompasses students whose 
skills cannot be described using these profi- 
ciency levels. The proficiency levels describe 
what students at each level can do and allow 
comparisons of the percentages of students 
in each country who perform at different lev- 
els of mathematics literacy (see technical 
notes in appendix A for more information 
about how levels were set). 

The U.S. average score of 483 on the com- 
bined mathematics literacy scale was just 
above the bottom cut point for level 3; the 
OECD average score of 500 was near the 
midpoint of level 3 (table 2, exhibit 5). The 
cutoff score of 607 for U.S. high performers 
(those in the top 10 percent in the United 
States) placed it just into level 5; the OECD 
score for high performers was near the mid- 
point of level 5. The cutoff U.S. score of 356 
for low performers (those in the bottom 10 
percent) was below level 1, while the OECD 
cutoff score of 369 for the bottom 10 percent 
was a level 1 score (figure 4, exhibit 5). 
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Figure 4. Distribution of combined mathematics literacy scores of 15-year-old students, by 
country: 2003 



Country 

OECD average 

OECD countries 

Finland 
Korea, Republic of 
Netherlands 
Japan 
Canada 
Belgium 
Switzerland 
Australia 
New Zealand 
Czech Republic 
Iceland 
Denmark 
France 
Sweden 
Austria 
Germany 
Ireland 
Slovak Republic 
Norway 
Luxembourg 
Poland 
Hungary 
Spain 
United States 
Portugal 

Italy 

Greece 

Turkey 

Mexico 

Non-OECD countries 

Hong Kong-China 
Liechtenstein 
Macao-China 
Latvia 

Russian Federation 
Serbia and Montenegro 
Uruguay 
Thailand 
Indonesia 
Tunisia 
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SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
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Exhibit 5. Description of proficiency levels for combined mathematics literacy: 2003 



Proficiency level 


Task descriptions 


Level 1 


At Level 1 students can answer questions involving familiar contexts where all rele- 
vant information is present and the questions are clearly defined. They are able to 
identify information and to carry out routine procedures according to direct instruc- 
tions in explicit situations. They can perform actions that are obvious and follow 
immediately from the given stimuli. 


Level 2 


At Level 2 students can interpret and recognize situations in contexts that require 
no more than direct inference. They can extract relevant information from a single 
source and make use of a single representational mode. Students at this level can 
employ basic algorithms, formula, procedures, or conventions. They are capable of 
direct reasoning and making literal interpretations of the results. 


Level 3 


At Level 3 students can execute clearly described procedures, including those that 
require sequential decisions. They can select and apply simple problem solving 
strategies. Students at this level can interpret and use representations based on 
different information sources and reason directly from them. They can develop short 
communications reporting their interpretations, results, and reasoning. 


Level 4 


At Level 4 students can work effectively with explicit models for complex concrete 
situations that may involve constraints or call for making assumptions. They can 
select and integrate different representations, including symbolic, linking them 
directly to aspects of real-world situations. Students at this level can utilize well- 
developed skills and reason flexibly, with some insight, in these contexts. They can 
construct and communicate explanations and arguments based on their interpreta- 
tions, arguments, and actions. 


Level 5 


At Level 5 students can develop and work with models for complex situations, iden- 
tifying constraints and specifying assumptions. They can select, compare, and eval- 
uate appropriate problem solving strategies for dealing with complex problems 
related to these models. Students at this level can work strategically using broad, 
well-developed thinking and reasoning skills, appropriate linked representations, 
symbolic and formal characterizations, and insight pertaining to these situations. 
They can reflect on their actions and formulate and communicate their interpreta- 
tions and reasoning. 


Level 6 


At Level 6 students can conceptualize, generalize, and utilize information based on 
their investigations and modeling of complex problem situations. They can link dif- 
ferent information sources and representations and flexibly translate among them. 
Students at this level are capable of advanced mathematical thinking and reason- 
ing. These students can apply this insight and understandings along with a mastery 
of symbolic and formal mathematical operations and relationships to develop new 
approaches and strategies for attacking novel situations. Students at this level can 
formulate and precisely communicate their actions and reflections regarding their 
findings, interpretations, arguments, and the appropriateness of these to the origi- 
nal situations. 



NOTE: In order to reach a particular level, a student must have been able to correctly answer a majority of items at that 
level. Students were classified into mathematics literacy levels according to their scores. Exact cut point scores are 
as follows: below level 1 (a score less than or equal to 357.77)\ level 1 (a score greater than 357.77 and less than or equal to 
420.07 ); level 2 (a score greater than 420.07 and less than or equal to 482.38 ); level 3 (a score greater than 482.38 and less 
than or equal to 544.68); level 4 (a score greater than 544.68 and less than or equal to 606.99); level 5 (a score greater than 
606.99 and less than or equal to 669.3); level 6 (a score greater than 669.3). 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2003. 



PISA 2003 Results From the U.S. Perspective 



The United States had greater percentages of 
students below level 1 and at levels 1 and 2 
than the OECD average percentages (figure 5, 
table B-6). The United States also had a lower 
percentage of students at levels 4, 5, and 6, 
than the OECD average percentages. This is 
somewhat different from the 2000 results, 
when reading literacy was the major domain. 
PISA 2000 results showed that while the per- 
centages of U.S. students performing at level 1 
and below were not measurably different from 
the OECD averages, the United States had a 
greater percentage of students performing at 
the highest level (level 5) compared to the 
OECD average (Lemke et al. 2001). 

In mathematics literacy in 2003, half (19) of the 
other 38 countries had a higher percentage of 
students at level 6 than the United States, 
including 16 OECD countries and 3 non-OECD 
countries (Hong Kong-China, Liechtenstein, 
and Macao-China) (figure 6, table B-6). In 
contrast, nine countries had a higher percent- 
age of students below level 1 than the United 



States (four of these nine — Greece, Italy, 
Mexico, andTurkey — were OECD countries). 
These same nine countries, as well as the 
Russian Federation and Portugal, had more 
students at level 1 than the United States. 

The United States had a lower percentage of 
students at level 6 than the OECD average for 
each of the four content area subscales 
( space and shape, change and relationships, 
quantity, and uncertainty) and a smaller per- 
centage than the OECD average at level 4 and 
level 5 on three of the four subscales (excep- 
tions were for uncertainty at level 5 and change 
and relationships at level 4) (tables B-7 
through B-10). 

The United States also had a higher percent- 
age of students at level 1 than the OECD 
average on each of the four subscales and 
more at level 2 for all subscales except uncer- 
tainty. On the quantity and uncertainty sub- 
scales, the United States also had greater 
percentages of students than the OECD aver- 
age percentages below level 1. 



Figure 5. Percentage distribution of 15-year-old students in the OECD countries and the 

United States on the combined mathematics literacy scale, by proficiency level: 2003 



Country 
OECD average - 
United States -\ 
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Below Level 1 O Level 1 d Level 2 Id Level 3 O Level 4 



80 100 



Level 5 ■ Level 6 



NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a majority of items at 
that level. Students were classified into mathematics literacy levels according to their scores. Exact cut point scores are as fol- 
lows: below level 1 (a score less than or equal to 357.77)\ level 1 (a score greater than 357.77 and less than or equal to 420.07 ); level 
2 (a score greater than 420.07 and less than or equal to 482.38)\ level 3 (a score greater than 482.38 and less than or equal to 544.68 ); 
level 4 (a score greater than 544.68 and less than or equal to 606.99 ); level 5 (a score greater than 606.99 and less than or equal to 
669. 3); level 6 (a score greater than 669.3). The Organization for Economic Cooperation and Development (OECD) average is the 
average of the national averages of the OECD member countries with data available. Detail may not sum to totals because of 
rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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Figure 6. Percentage distribution of 15-year-old students on the combined mathematics 
literacy scale, by proficiency level and country: 2003 

Country 

OECD average 

OECD countries 

Finland 
Canada 
Korea 
Netherlands 
Australia 
Iceland 
Ireland 
Denmark 
Japan 
New Zealand 
Switzerland 
Czech Republic 
Sweden 
France 
Austria 
Slovak Republic 
Poland 
Norway 
Belgium 
Luxembourg 
Hungary 
Spain 
Germany 
United States 
Portugal 
Italy 
Greece 
T urkey 
Mexico 

Non-OECD countries 

Macao-China 
Hong Kong-China 
Liechtenstein 
Latvia 

Russian Federation 
Serbia and Montenegro 
Thailand 
Uruguay 
Indonesia 
Tunisia 

0 20 40 60 80 100 

Percent 

| Below level 1 □ Level 1 □ Levels 2, 3, 4, and 5 | Level 6 

NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a majority of items at 
that level. Students were classified into mathematics literacy levels according to their scores. Exact cut point scores are as fol- 
lows: below level 1 (a score less than or equal to 357.77 ); level 1 (a score greater than 357.77 and less than or equal to 420.07)', level 
2 (a score greater than 420.07 and less than or equal to 482.38)] level 3 (a score greater than 482.38 and less than or equal to 544.68)] 
level 4 fa score greater than 544.68 and less than or equal to 606.99)] level 5 fa score greater than 606.99 and less than or equal to 
669.3)] level 6 fa score greater than 669.3).Vr\e Organization for Economic Cooperation and Development (OECD) average is the 
average of the national averages of the OECD member countries with data available. Because PISA is principally an OECD 
study, the results for non-OECD countries are displayed separately from those of the OECD countries and are not included in 
the OECD average. Due to low response rates, data for the United Kingdom are not discussed in this report. Detail may not 
sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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Changes in Mathematics Literacy 
Performance From 2000 to 2003 

Because mathematics literacy was a minor 
domain in 2000, items from only two content 
areas ( space and shape and change and rela- 
tionships) were administered in that assess- 
ment cycle. As a result, it is not possible to 
describe changes since 2000 for the combined 
mathematics literacy scale or for the other 
two content areas ( quantity or uncertainty). 
Rather, changes can only be discussed for 
the two content areas represented in 2000 and 
2003 ( space and shape and change and rela- 
tionships). Data from 2000 were re-scaled 
using 2003 mathematics literacy data in order 
to make these comparisons. 6 Comparisons 
were available only for OECD countries com- 
mon to both the 2000 and 2003 cycle (28 coun- 
tries) but results for the United Kingdom and 
the Netherlands are not discussed here due 
to low response rates for the United Kingdom 
in 2003 and the Netherlands in 2000. In total, 
results for 26 OECD countries were available 
for comparisons and are discussed here. 

There were no measurable changes in the 
U.S. scores from 2000 to 2003 on either the 
space and shape subscale or the change and 
relationships subscale (table B-11). In both 
2000 and 2003, about two-thirds of the other 
countries outperformed the United States 
on these scales. Eighteen of the other 25 
OECD countries outscored the United 
States on the space and shape scale in 2003 
(compared to 19 in 2000); 17 OECD countries 
outscored the United States on the change 
and relationships scale in 2003 (compared to 
14 in 2000). 

Five countries had their scores improve on 
the space and shape subscale. Four of the 
five countries with improved scores on the 
space and shape subscale also showed 
improvements on the change and relation- 
ships scale (Belgium, Czech Republic, 
Luxembourg, and Poland; Italy improved its 
score on the space and shape scale but not 
on the change and relationships scale). 



Of the five countries that showed increases 
on the space and shape subscale, Belgium 
and the Czech Republic already outper- 
formed the United States in 2000 and also 
improved their scores in 2003. Italy, despite 
its improvement in score, was not measura- 
bly different from the United States in either 
year. Poland, which was not measurably dif- 
ferent from the United States in 2000, 
outscored the United States in 2003, and 
Luxembourg, which scored below the United 
States in 2000, also outscored the United 
States in 2003. 

Two countries (Mexico and Iceland) showed 
decreased scores from 2000 to 2003 on the 
space and shape scale. Despite these 
decreases in performance, there was no 
change in the relative position of either 
country compared to the United States: that 
is, Iceland outperformed the United States 
in 2000 and 2003 on the space and shape sub- 
scale, and Mexico performed worse than the 
United States in 2000 and 2003. 

Of the other 25 OECD countries, 11 had their 
scores improve from 2000 to 2003 on the 
change and relationships subscale, while no 
country had a decrease. Of the 11 countries 
that improved from 2000 to 2003, several 
already outperformed the United States in 
2000: Belgium, Canada, Denmark, Finland, 
and Korea all scored higher than the United 
States in 2000 on the change and relation- 
ships subscale. Several other countries 
were not measurably different from the 
United States in 2000, but outperformed the 
United States in 2003 (Czech Republic, 
Germany, Hungary). Three countries 
(Luxembourg, Poland, and Spain) had lower 
scores than the United States in 2000 on the 
change and relationships subscale, but were 
not measurably different from the United 
States in 2003. Portugal, despite its 
improvement in score, still scored lower than 
the United States in 2000 and 2003. 



6 For more information on scaling, see the technical notes in appendix A. 
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U.S. Performance in 
Problem Solving 

As noted, one of PISA’s major goals is to 
assess skills that cut across traditional cur- 
ricular areas. In 2003, PISA assessed stu- 
dents’ abilities in problem solving. 7 

Problem solving is defined as: 

...an individual's capacity to use cognitive 
processes to confront and resolve real, 
cross-disciplinary situations where the 
solution is not immediately obvious, and 
where the literacy domains or curricular 
areas that might be applicable are not with- 
in a single domain of mathematics, sci- 
ence, or reading. (OECD 2003, p. 156). 

Students completed exercises that assessed 
their capabilities in using reasoning 
processes not only to draw conclusions but 
to make decisions, to troubleshoot (i.e., to 
understand the reasons for malfunctioning 
of a system or device), or to analyze 
the procedures and structures of a 
complex system (such as a sim- 
ple kind of programming lan- 
guage). Problem-solving 
items required students to 
apply various reasoning 
processes, such as induc- 
tive and deductive reasoning, 
reasoning about cause and 
effects, or combinatorial rea- 
soning (i.e., systematically compar- 
ing all the possible variations which can 
occur in a well-described situation). 
Students were also assessed in their skills 
in working toward a solution and communi- 
cating the solution to others through appro- 
priate representations. Sample problem- 
solving items and student responses are 
shown here. 



For more information about the problem- 
solving framework, please refer to The PISA 
2003 Assessment Framework: Mathematics, 
Reading, Science, and Problem Solving 
Knowledge and Skills (OECD 2003). 
Additional released problem-solving items 
can be found at 

http://nces.ed.gov/survevs/pisa . 
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7 PISA 2003's problem-solving assessment focused explicitly on problem-solving skills, using a variety of contexts, disciplines, and 
problem types. The items used to measure problem solving in PISA 2003 were different from other items, such as those measuring 
mathematics literacy. Problem solving can also be embedded within measures of content areas such as mathematics or science, 
however. TIMSS 2003, for example, incorporated an explicit aspect of problem solving and inquiry into the description of desired 
outcomes for mathematics and science. A review of mathematics and science items in PISA andTIMSS showed that 38 percent of 
eighth-gradeTIMSS 2003 mathematics items and 48 percent of PISA 2003 mathematics literacy items measured some aspect of 
problem solving; additionally, 26 percent of eighth-gradeTIMSS 2003 science items and 49 percent of PISA science literacy items 
measured problem-solving skills (Dossey, O'Sullivan, and McCrone forthcoming). 
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Exhibit 6. Problem-solving sample item 1: 2003 



Design By Numbers © 1 



Design by Numbers is a -design tool lor generating graphics on compilers Pictures can 
be generated by giving a Sfll of commands to the program 



Study carefully the fallowing example commands and pictures before answering the 
questions. 




Papm d 
Pnr 1-TO 
LmtMOMM 





Papflr 1W 
Pond 

Lille 2v LU UU 2i 
UiHfSOaO 5C80 

l.iiii:- fr" HU :*!:■ l v .l 




Question 4: DESIGN BY NUMBERS 

Which of the following commands generated the graphic shown below? 

A Paper 0 
B) Paper 20 
C Paper 50 
D Paper 75 







RESULTS: 

U S percent correct 41 .3 

OECD percent correct 50,3 

item diflicuily Level 2 

Problem Type: System analysis and design 



SOURCE: Organization for Economic 
Cooperation and Development (OECD), 
Program for International Student 
Assessment (PISA), 2003. 



'Design by Numbers was developed by the Aesthetics and Computation Group at the MIT Media Laboratory. 
Copyright 1999, Massachusetts Institute ofTechnology.The program can be downloaded from 
ht tp://dbn.media.mit.edu. 
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Exhibit 6. Problem-solving sample item 1: 2003 — Continued 




Question 8: DESIGN BY NUMBERS 

The followrng shows an example ol the "Repeat' command. 



pwo 

Pen IDO 
Repeal A»!IL 



Lx« P0A4C-* 




The command ‘Repeat A M 90" tells the 
program to repeal She actions m brackets { 't. 
tor successive values of A from A=50 to 

a-bg. 



Write commands to generate the following graphic. 
Palmer O 

P&\ too 
fi.cjpca-t A O 40 

Line 2JD A tO A 




RESULTS: 

U ,$ percent correct r ... 3 1 .4 

OECD percent correct 39.6 

Hem difficulty: 12 

Problem Type System analysis and design 



SOURCE: Organization for Economic 
Cooperation and Development (OECD), 
Program for International Student 
Assessment (PISA), 2003. 
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Exhibit 7. Problem-solving sample item 2: 2003 



Irrigation 



Below is a diagram of a system of irrigation channels lor watering sections of crops. The 
gates A to H can be opened and dosed lo let the water go where it is needed When a 
gate is dosed no water can pass th rough it. 

This is a problem aboul finding a gate which is stuck dosed, preventing water From 
Flowing through Ihe system of channels. 

Figure 1: A system of irrigation channels 




Michael nonces mel me waier is not always going where it is supposed to. 

He thinks that one ol the gates is stuck closed, so lhal when it «s switched lo "open", it 
does not open. 



Question 17: IRRIGATION 

Michael uses the settings given in Table 1 to test the gates. 



tfjjflaewi -(Mj 



A 




C 


O 


E 


F 


G 


H 


Open 


Closed 


Open 


Open 


Closed 


Open 


Closed 


Open 



With the gate settings as given in Table 1, On the diagram below draw all the possible 
paths tor the (low ol water Assume that all gates are working according to the sellings 



In 




RESULTS: 

U.S.. percent correct 

0 EC D percenl correct 62 . 9 

Hem diHicufty; Level i 
Problem Type; Trouble Shooting 



SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2003. 
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Exhibit 7. Problem-solving sample item 2: 2003 — Continued 



Irrigation 



Question IB: IRRIGATION 

Michael finds lhat, when the gates have the Table 1 sellings, no water News through, 
indicating that at least one of the gates get to ■'open" i$ slue*: closed. 

Decide for each problem case below whether ihe water will flow through all ihe way, 
Circle “Ye§" or “No' in each ease. 



Problem Case 


Will water flow through all the way? 


“Sate A is Stuck closed Ail otFiir gates 
are working properly as set in Table 1 . 


Yes /© 


Gate D *s stuck dosed All other gales 


(Yes) f Ho 


are working property as set in Table 1 . 


Gate F is stuck closed. Ail other gates are 
working properly as sal in Table i . 


(Yes)/ No 



RESULTS: 

US. percent correct 44.6 

OECD percent correct 5T.3 

item difficulty: Level 2 

Problem Type' Trouble Shooing 



Question 1 9: 1 RRlG ATIO N 

Michael wants to be able to test whether gate o is slue* closed. 

In Ihe Pul lowing table, show sellings for the gales to tegl whether gate □ 1$ sluck CJ0$ed 
when it :SSHl hr "open" 



Seftfnsrs for gates (each one “open” or "dosed”} 



A 


B 


c 


D 


E 


F 


G 


H 


open 


dosed 


dosed 


open 


dosed 


dosed 


dosed 


dosed 



RESULTS: 




U.S. percent correct 


..,51.5 


OECD percent correct ,,, ,, 


, .. 54.4 


item difficulty; Level 2 




Problem Type' Trouble Shooting 
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SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2003. 
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Exhibit8. Problem-solving sample item 3: 2003 



Library System 



The John Hcbson High ScIidoI library ho$ a simple system lor lendi rig t*x>k&: tar staff 
members the loan per»«i Is 2$ days, and far students the loan period >s i days- The 
igNowing is a decision tree digram showing this simple system; 




The Greenwood High School library has a similar, Out more complicated, lending 
system: 

* All publications classified as "Reserved" have a loan period of 2 days. 

* For boohs (not including magazines) Inat are not on the reserved list, the toan period 
is £8 days for staff, and 1 4 days for students 

* For magazines that are not on Ihe reserved list, the toan period is 7 days for 
everyone. 

* Persons with any overdue items are not allowed to borrow anything. 

Question 13: LIBRARY SYSTEM 

You are a student at Greenwood High School, and you do not have any overdue iiem$ 
from the library. You want 1o borrow a hook that is not on the reserved list How long 
can you borrow (he book for? 

I 4- 

Answer; .71 days 



RESULTS: 




Lf.S. percent correct 


75.3 


OECD percent correct.. 
Hem difficutty: Level 1 


74 .a 


Problem Type' System analysis and design 



SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2003. 
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Exhibit8. Problem-solving sample item 3: 2003 — Continued 



Library System 



Question 14: LIBRARY SYSTEM 



X4a2QG2-61 02 It 12 21 22 23 Jr SB 



Develop a decision tree diagram for ihe Greenwood High School Library system so 
that an automated checking system can be designed to deal with book and magazine 
loans at the library. Your checking system should be as efficient as possible (is. it 
should hava Ihe least number of checking steps). Note that each checking slap should 
have only two outcomes and the outcomes should be labeled appropriately (e.g/Yes" 
and "No'). 



START 



I 




5 


jBll not; 


■ ^ 


fiervTcrvj ijn.-f 



loan pcjnod 
I'l 2- dlfi 



kun period 

IS T 



roen pmoa 
Is JJJ an** 




rno 



loan period 
is I 4 days 



RESULTS: 

U.S. percent correcl 13.5 

OECD percent correct 14.3 

Item difficulty: Level 3 

Problem Type System analysis and design 



SOURCE: Organization for 
Economic Cooperation and 
Development (OECD), Program for 
International Student Assessment 
(PISA), 2003. 
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Problem-solving scores are reported 
on a scale with a mean of 500 and 
standard deviation of 100. Fifteen- 
year-old students in the United 
States had an average score of 477 
on the problem-solving scale, lower 
than the OECD average score of 500 
(table 3, table B-12). U.S. students 
scored lower in problem solving 
than their peers in 25 of the other 38 
countries (22 OECD and 3 non- 
OECD countries). Eight countries (3 
OECD — Greece, Mexico, and 
Turkey — and 5 non-OECD countries) 
reported lower scores compared to 
the United States in problem solv- 
ing. Three OECD country scores 
(and two non-OECD country scores) 
were not measurably different from 
the U.S. average score in problem 
solving. 

On average, U.S. high achievers for 
problem solving (those scoring in 
the top 10 percent in the United 
States) were outperformed by their 
OECD counterparts (figure 7, table 
B-13). To be in the top 10 percent of 
students in the United States, stu- 
dents needed at least a score of 604, 
while they needed a score of 446 or 
better inTunisia but 675 or better in 
Japan. Low performers in the United 
States (those in the bottom 10 per- 
cent) scored 347 or lower, which was 
lower than the cutoff score of 368 or 
lower for the OECD average. There 
was approximately a 256 point score 
difference, or two and a half stan- 
dard deviations, between the cutoff 
scores for the top 10 percent (604) 
and the bottom 10 percent (347) of 
15-year-old students for problem 
solving in the United States. 



Table 3. Average scores of 15-year-old students on 
the problem-solving scale, by country: 2003 

Country 



OECD average | 500 



OECD countries 



Korea 550 



Finland 548 



Japan | 547 



New Zealand 533 



Australia 530 



Canada 529 



Belgium | 525 



Switzerland | 521 



Netherlands | 520 



France 519 



Denmark 517 



Czech Republic | 516 



Germany | 513 



Sweden 509 



Austria 506 



Iceland 505 



Hungary | 501 



Ireland 498 



Luxembourg | 494 



Slovak Republic | 492 



Norway 490 



Poland 487 1 



Spain 482 

United States 477 

Portugal 470 

Italy 470 



Greece 


449 


Turkey 


408 


Mexico 


384 



Non-OECD countries 



Hong Kong-China 


548 


Macao-China 


532 


Liechtenstein 


529 



Latvia 483 

Russian Federation 479 



Thailand 


425 


Serbia and Montenegro 


420 


Uruguay 


411 


Indonesia 


361 


Tunisia 


345 



□ Average is significantly higher than the U.S. average 

□ Average is not significantly different than the U.S. average 
■ Average is significantly lower than the U.S. average 

NOTE: Statistical comparisons between the U.S. average and the 
Organization for Economic Cooperation and Development (OECD) aver- 
age take into account the contribution of the U.S. average toward the 
OECD average. The OECD average is the average of the national aver- 
ages of the OECD member countries with data available. Because the 
Program for International Student Assessment (PISA) is principally an 
OECD study, the results for non-OECD countries are displayed separate- 
ly from those of the OECD countries and are not included in the OECD 
average. Due to low response rates, data for the United Kingdom are not 
discussed in this report. 

SOURCE: Organization for Economic Cooperation and Development 
(OECD), Program for International Student Assessment (PISA), 2003. 
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Figure 7. Distribution of problem-solving scores of 15-year-old students, by country: 2003 



Country 

OECD average 

OECD countries 

Korea - 
Finland - 
Japan - 
New Zealand - 
Australia - 
Canada - 
Belgium - 
Switzerland - 
Netherlands - 
France - 
Denmark - 
Czech Republic - 
Germany - 
Sweden - 
Austria - 
Iceland - 
Hungary - 
Ireland - 
Luxembourg - 
Slovak Republic - 
Norway - 
Poland ■ 
Spain ■ 
United States - 
Portugal - 
Italy - 
Greece - 
Turkey - 
Mexico - 

Non-OECD countries 

Hong Kong-China - 
Macao-China - 
Liechtenstein - 
Latvia - 
Russian Federation - 
Thailand - 
Serbia and Montenegro - 
Uruguay - 
Indonesia - 
Tunisia - 
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NOTE:The Organization for Economic Cooperation and Development (OECD) average is the average of the national averages of the OECD 
member countries with data available. Because the Program for International Student Assessment (PISA) is principally an OECD study, the 
results for non-OECD countries are displayed separately from those of the OECD countries and are not included in the OECD average. Due to 
low response rates, data for the United Kingdom are not discussed in this report. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2003. 
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Of the 38 other participating countries, 22 
countries (including 16 OECD countries) had 
less variation (as measured by standard 
deviation) in performance in problem solving 
than the United States, while 3 countries 
(Belgium, Japan, and Uruguay) showed 
greater variation in performance (table B- 
14). The U.S. variation in performance was 
not measurably different from the OECD 
average variation. 



Along with scale scores, PISA 2003 also 
uses three proficiency levels (levels 1 
through 3, with level 3 being the highest level 
of proficiency) to describe student perform- 
ance in problem solving. An additional level 
(below level 1) encompasses students 
whose skills cannot be described using 
these proficiency levels (exhibit 9). The pro- 
ficiency levels describe what students at 
each level can do and allow comparisons of 
the percentages of students in each country 
who performed at different levels in problem 
solving (see appendix A for more informa- 
tion about how levels were set). 



Exhibit 9. Description of proficiency levels for problem solving: 2003 



Proficiency level 


Task descriptions 


Level 1 


At Level 1 students can solve problems where they have to deal with a single data 
source containing discrete, well-defined information. They understand the nature of a 
problem and consistently locate and retrieve information related to the major fea- 
tures of the problem. Level 1 students may be able to transform the information in the 
problem to present the problem differently (e.g., take information from a table to cre- 
ate a drawing or graph). Also, students may be able to apply information to check a 
limited number of well-defined conditions within the problem. However, Level 1 stu- 
dents are generally incapable of dealing with multi-faceted problems involving more 
than one data source or requiring the student to reason with the information provided. 


Level 2 


At Level 2 students use reasoning and analytic processes and solve problems 
requiring decision-making skills. Level 2 students apply various types of reason- 
ing (inductive and deductive reasoning, reasoning about causes and effects, or 
combinatorial reasoning, that is, systematically comparing all possible variations 
in well-described situations) to analyze situations and to solve problems that 
require students to make a decision among well-defined alternatives. To analyze a 
system or make decisions, Level 2 students combine and synthesize information 
from a variety of sources. Students may need to combine various forms of repre- 
sentations (e.g., a formalized language, numerical information, and graphical 
information), handle unfamiliar representations (e.g., statements in a proto-pro- 
gramming language or flow diagrams related to a mechanical or structural 
arrangement of components), or draw inferences based on two or more sources of 
information. 


Level 3 


At Level 3 students do not only analyze a system and make decisions, they also 
represent the underlying relationships in a problem and relate these to the solu- 
tion. Level 3 students approach problems systematically, construct their own rep- 
resentations and verify that their solution satisfies all requirements of the problem. 
These students communicate their solutions to others using written statements 
and other representations. 



NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a majority 
of items at that level. Students were classified into problem-solving levels according to their scores. Exact cut point 
scores are as follows: below level 1 (a score less than or equal to 404.06)\ level 1 (a score greater than 404.06 and less 
than or equal to 498.08); level 2 (a score greater than 498.08 and less than or equal to 592. 10); level 3 (a score greater 
than 592. 10). 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2003. 
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The U.S. average score of 477 on the prob- 
lem-solving scale placed it at level 1, while 
the OECD average score was at level 2 
(table B-12, exhibit 9). The cutoff score of 
604 for U.S. high performers (those in the top 
10 percent in the United States) equated to 
a level 3 score, while the U.S. cutoff score of 
347 for low performers (those in the bottom 
10 percent) was below level 1 (table B-13, 
exhibit 9). 



Twenty-four percent of U.S. students scored 
below level 1, 34 percent at level 1, 30 percent 
at level 2, and 12 percent at level 3 (figure 8, 
table B-15). The United States had greater 
percentages of students below level 1 and at 
level 1 than the OECD average percentages. 
The United States also had a lower percent- 
age of students at levels 2 and 3 than the 
OECD average percentages. Four countries 
(Finland, Hong Kong-China, Japan, and 
Korea) had 30 percent or more of their stu- 
dents performing at level 3 in problem solv- 
ing, compared with 12 percent for the United 
States and 18 percent for the OECD average. 



Figure 8. Percentage distribution of 15-year-old students in the OECD countries and the 
United States on the problem-solving scale, by proficiency level: 2003 



Country 
OECD average 

United States 




Percent 



| Below level 1 □ Level 1 □ Level 2 



Level 3 



NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a majority of 
items at that level. Students were classified into problem-solving levels according to their scores. Exact cut point scores 
are as follows: below level 1 (a score less than or equal to 404.06)', level 1 (a score greater than 404.06 and less than or equal 
to 498.08)\ level 2 (a score greater than 498.08 and less than or equal to 592. 10)', level 3 (a score greater than 592. 10). The 
Organization for Economic Cooperation and Development (OECD) average is the average of the national averages of the 
OECD member countries with data available. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2003. 
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U.S. Performance in 
Reading Literacy and 
Science Literacy 

Of the 41 countries that participated in PISA 
2003, 32 also participated in PISA 2000. 
Changes in reading literacy and science lit- 
eracy are reported for 29 of these 32 coun- 
tries. 8 

In 2003, the average U.S. score in reading 
literacy was 495, not measurably different 
from the OECD average of 494 (figure 9, 
table B-16). Eleven countries (including 
9 OECD countries) among the other 38 
countries outperformed the United States 
in reading literacy in 2003. 



There was no measurable change in either 
the U.S. reading literacy score from 2000 to 
2003 or the U.S. position compared to the 
OECD average, although scores in 12 other 
countries did change (table B-16). 9 Four 
countries saw their average reading literacy 
scores increase (two non-OECD countries, 
Latvia and Liechtenstein, and two OECD 
countries, Luxembourg and Poland). The 
United States outperformed all four of these 
countries in 2000; in 2003, scores for Latvia 
and Poland were not measurably different 
from the U.S. scores in reading literacy, 
while Liechtenstein outscored the United 
States in 2003. Despite an increase in 
Luxembourg’s average reading literacy 
score, the United States outperformed it in 
2000 and 2003. 



Figure 9. Average reading literacy and science literacy scores of 15-year-old students in the 
OECD countries and the United States: 2003 



Average 

score 

700 



600 



500 

400 

300 

200 

100 

0 




500 4g-|* 




Science literacy 



I OECD average 



I I United States 



* Average is significantly different from OECD average. 

NOTE:The Organization for Economic Cooperation and Development (OECD) average is the average of the national averages 
of the OECD member countries with data available. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 



8 Due to low response rates, data for the Netherlands were not discussed for PISA 2000; data for PISA 2003 for the United Kingdom 
are also not discussed due to low response rates; data for Brazil were not available at the time of production for this report. 

9 Large standard errors for the United States in 2000 may account at least in part for the fact that U.S. reading literacy and science lit- 
eracy scores were not measurably different from 2000 to 2003 and that the scores were not different from the OECD averages in 2000. 
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Eight countries’ scores (including seven 
OECD countries) were lower in 2003 than 
2000 in reading literacy. Decreases in two of 
these eight countries’ scores resulted in a 
change relative to the United States. Japan, 
which outperformed the United States in 
reading literacy in 2000, was not measurably 
different in 2003, while Spain, which did not 
perform measurably differently in 2000, per- 
formed worse than the United States in 2003. 

In 2003, the U.S. average score in science 
literacy was 491, lower than the OECD aver- 
age of 500 (figure 9, table B-17). Eighteen 
countries (including 15 OECD countries) 
outscored the United States in science in 
2003. 

There was no measurable difference 
between the U.S. average science literacy 
score of 499 in 2000 and 491 in 2003, although 
the relative position of the United States 
compared to the OECD average did change 
(the U.S. science literacy score in 2000 was 
not measurably different from the OECD 
average, while in 2003 the U.S. score was 
below the OECD average). Seventeen 
countries showed changes in their 
scores from 2000 to 2003 — 5 coun- 
tries (all OECD countries) had 
lower scores in 2003 than in 
2000 and 12 countries (includ- 
ing 9 OECD countries) had 
higher scores (table B-17). 

The OECD average score in 
science literacy was 500 in 
2000 and 2003. 

Of the 12 countries whose science litera- 
cy scores improved between 2000 and 2003, 

8 also improved their performance relative to 
the United States. Belgium, the Czech 
Republic, France, Germany, Liechtenstein, 
and Switzerland did not perform differently 
from the United States in 2000 but outscored 



the United States in 2003. Latvia and the 
Russian Federation scored below the U.S. 
average in 2000 but were not measurably dif- 
ferent in 2003. Of the five countries whose 
science literacy scores decreased between 
2000 and 2003, two (Canada and Korea) con- 
tinued to outperform the United States, one 
(Norway) was not measurably different in 
either year, one (Mexico) performed 
below the U.S. average in both 
years, and one (Austria) went 
from outscoring the United 
States to not being measur- 
ably different from the 
United States. 
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Differences in 
Performance by 
Selected Student 
Characteristics 

This section provides information about how 
students with various characteristics (males 
and females, students of different races and 
from different socioeconomic backgrounds) 
performed on PISA 2003. Because PISA 
2003’s emphasis was on mathematics litera- 
cy and problem solving, the focus in this 
section is on performance in these areas. 10 
This report does not address possible 
changes in performance for these groups 
from 2000 to 2003. 

When considering these results, it is impor- 
tant to bear in mind that there need not be a 
cause-and-effect relationship between being 
a member of a group and achievement in 
PISA 2003. Student performance can be 
affected by a complex mix of educational 
and other factors that are not examined 
here. 

Sex 

Fifteen-year-old females in the United 
States scored 480 on the combined mathe- 
matics literacy scale, which was lower than 
the average male score of 486 (figure 10, 
table B-18). Males also outperformed 
females in 25 other countries (20 OECD 
countries and 5 non-OECD countries), a pat- 
tern evident in the OECD average scores of 
494 for females and 506 for males. Iceland 
was the only country in which females 
scored higher in mathematics literacy than 
males. 

Within the United States, greater percent- 
ages of male students performed at level 6 
(the highest level) than female students in 



mathematics literacy, but larger percentages 
of females were not seen at lower levels 
(below level 1 and levels 1 through 5, table B- 
19). In other words, differences in the overall 
scores between males and females in the 
United States were due at least in part to 
the fact that a greater percentage of males 
were found among the highest performers, 
not to a greater percentage of females found 
among the lowest performers. 

On average across the OECD countries, 
males outperformed females on each of the 
four mathematics literacy subscales (table 
B-20). In the United States, differences 
between males and females were evident 
only on the space and shape subscale. 

In the majority of the PISA 2003 countries 
(32 out of 39 countries), including the United 
States, there were no measurable differ- 
ences in problem-solving scores by sex 
(figure 10, table B-21). However, females 
outscored their male peers in problem solv- 
ing in six of the remaining seven participat- 
ing countries (including four OECD coun- 
tries), as well as at the OECD average. 

Males outscored females in problem solving 
in Macao-China. 

As in 2000, females in the United States and 
nearly every other participating country 
outscored males in reading literacy in 2003 
(table B-22). Only Liechtenstein showed no 
statistical difference between males and 
females in 2003, although there was a differ- 
ence in favor of females in 2000. 

There was no measurable difference 
between the performance of U.S. males and 
females in science literacy in PISA 2000 or 
PISA 2003, and scores for neither group 
changed between 2000 and 2003. Thirteen 
countries showed differences between 
males and females in 2003 (12 OECD coun- 
tries and the Russian Federation). Eleven of 
the 13 countries showed differences in favor 
of males, but in Finland and Iceland females 
outperformed males. 



10 lnformation on performance in reading literacy and science literacy by sex and race/ethnicity is provided, however. 
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Figure 10. Differences in average scores of 15-year-old students on the combined 

mathematics literacy scale and in problem solving, by sex and country: 2003 
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Australia ■ 
Austria ■ 
Belgium ■ 
Canada ■ 
Czech Republic ■ 
Denmark ■ 
Finland ■ 
France ■ 
Germany ■ 
Greece ■ 
Hungary ■ 
Iceland ■ 
Ireland ■ 
Italy ■ 
Japan ■ 
Korea 

Luxembourg ■ 
Mexico ■ 
Netherlands ■ 
New Zealand ■ 
Norway ■ 
Poland ■ 
Portugal ■ 
Slovak Republic ■ 
Spain - 
Sweden ■ 
Switzerland ■ 
T urkey ■ 
United States 
Non-OECD countries 

Hong Kong-China ■ 
Indonesia ■ 
Latvia ■ 
Liechtenstein ■ 
Macao-China ■ 
Russian Federation ■ 
Serbia and Montenegro - 
Thailand ■ 
T unisia ■ 
Uruguay ■ 
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I Average score difference is statistically significant 
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NOTE: Each bar above represents the average score difference between males and females on combined mathematics literacy 
and problem solving. The Organization for Economic Cooperation and Development (OECD) average is the average of the 
national averages of the OECD member countries with data available. Because the Program for International Student 
Assessment (PISA) is principally an OECD study, the results for non-OECD countries are displayed separately from those of 
the OECD countries and are not included in the OECD average. Due to low response rates, data for the United Kingdom are not 
discussed in this report. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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Socioeconomic Status 

The measure of student socioeconomic status 
(SES) used in PISA 2003 is based on the 
occupational status of the student’s father or 
mother (whichever was higher) as reported by 
the student. Parental occupation was coded 
based on the International Standard 
Classification of Occupations (ISCO) 
(International Labor Organization 1990). 
Occupational codes were in turn mapped onto 
an internationally comparable index of occupa- 
tional status, the International Socioeconomic 
Index (ISEI), developed by Ganzeboom, De 
Graaf, andTreiman (1992). Using the index, 
students were assigned numbers ranging from 
about 16 to 90 based on their parents’ occupa- 
tions, so that they were arrayed on a continu- 
um from low to high socioeconomic status, 
rather than placed into discrete categories. 
Typical occupations among parents of 15-year- 
olds with between 16 and 35 points on the ISEI 
scale include small-scale farmer, metalworker, 
mechanic, taxi or truck driver, and waiter/wait- 
ress. Between 35 and 53 index points, the most 
common occupations are bookkeeping, sales, 
small business management, and nursing. As 
the required skills increase, so does the status 
of the occupation. Between 54 and 70 points, 
typical occupations are marketing manage- 
ment, teaching, civil engineering, and account- 
ant. Finally, between 71 and 90 points, the top 
international quarter of the index, occupations 
include medicine, university teaching, and law 
(OECD 2001). 

The average ISEI index score for the United 
States in 2003 was 55, higher than that of all 
but two countries (Norway and Iceland) (table 
B-23). Low ISEI students in the United States 
were also comparatively better off in terms of 
socioeconomic status than most of their 
OECD peers. U.S. students with low ISEI 
(those in the bottom 25 percent in the United 
States) had an average index value of 33, 
which was higher than the index values for low 
ISEI students in 35 of the other 38 PISA 2003 
countries (including 25 OECD countries). Two 
countries (Japan and Norway) reported higher 
average index values for low ISEI students 
compared to the United States. 



Within the United States, students with low 
ISEI values were outperformed in mathemat- 
ics literacy by their peers with higher ISEI 
values (table B-24). Moreover, U.S. students 
with low ISEI values were outperformed by 
their peers with low ISEI values in 22 of the 
39 PISA 2003 countries (including 18 OECD 
countries) for mathematics literacy. 

Students with the highest ISEI background 
in the United States (those in the top quar- 
ter) were outperformed by high ISEI stu- 
dents from 20 other countries (including 19 
OECD countries) in mathematics literacy. 

The overall linkage of ISEI to mathematics 
literacy and problem solving can be exam- 
ined by the specific change in score on the 
combined mathematics literacy scale in 
response to a one standard deviation change 
in the ISEI index score for each country. A 
greater increase in the average achievement 
score in a country implies a stronger rela- 
tionship between socioeconomic status and 
performance in that country. 

For example, in the United States, a one 
standard deviation change in the ISEI index 
was associated with an average difference of 
30 points on the combined mathematics liter- 
acy and 31 points on the problem-solving 
scale (table B-25). In Macao-China, socio- 
economic background differences in achieve- 
ment were at a minimum — one standard devi- 
ation’s difference on the ISEI index was 
associated with a 10 point difference on the 
combined mathematics literacy scale and a 
12 point difference on the problem-solving 
scale. By contrast, among students in 
Hungary, a one standard deviation change in 
ISEI score was associated with about a 41 
point difference in both mathematics literacy 
and problem-solving achievement scores. 
Twelve countries (including six OECD coun- 
tries) had a weaker relationship between 
ISEI and problem-solving performance than 
the United States, while three countries 
(Belgium, Germany, and Hungary) had a 
stronger one. Belgium, Germany, and 
Hungary also had stronger relationships 
between ISEI and mathematical literacy than 
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the United States, as did the Czech Republic 
and Poland. Eleven countries (including 6 
OECD countries) had weaker relationships. 

Race/Ethnicity 

Racial and ethnic groups vary between coun- 
tries, so it is not possible to compare their per- 
formance across countries on international 
assessments.Thus, this section refers only to 
2003 findings for the United States. Throughout 
this section, "White” refers to White, non- 
Hispanic students, “Black” to Black, non- 
Hispanic students, "Asian” to Asian, non- 
Hispanic students, and "Hispanic” to Hispanic 
students of any race. Results for two groups 
(American Indian or Alaska Native and 
Hawaiian or Other Pacific Islander) are not 
shown separately because small sample sizes 
did not allow for accurate estimates. 

In both mathematics literacy and problem 
solving, Blacks and Hispanics scored lower, 
on average, than Whites, Asians, and stu- 



dents of more than one race (figure 11, table 
B-26). Hispanic students, in turn, outscored 
Black students. This pattern of performance 
on PISA 2003 by race/ethnicity is similar to 
that found in PISA 2000 and on the National 
Assessment of Educational Progress 
(NAEP) (Braswell, Daane, and Grigg 2003; 
Lemke et al. 2001). 

In both mathematics literacy and problem 
solving, the average scores for Blacks and 
Hispanics were below the respective OECD 
average scores, while scores for Whites 
were above the OECD average scores. 
Students who wereWhite, Asian, and of 
more than one race scored at level 3 in 
mathematics literacy, compared to level 2 for 
Hispanic students and level 1 for Black stu- 
dents (figure 11, exhibit 5). In problem solv- 
ing, average scores for Whites and Asians 
placed them in level 2, while Black, Hispanic, 
and students of more than one race scored at 
level 1 (figure 11, exhibit 9). 



Figure 11. Average scores of U.S. 15-year-old students on the combined mathematics literacy 
scale and in problem solving, by race/ethnicity: 2003 



Average 

score 

700 



600 




OECD White Asian More Hispanic Black OECD White Asian More Hispanic Black 

average than average than 

one one 

race race 

Race/ethnicity 

I Mathematics literacy □ Problem solving 



* Average is significantly different from OECD average. 

NOTE: Reporting standards not met for American Indian/Alaska Native and Native Hawaiian/Other Pacific Islander. Black 
includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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For Further Information 

This report provides selected findings from 
PISA 2003 from a U.S. perspective. Readers 
may be interested in exploring other aspects 
of PISA’s results. Additional findings are 
presented in the OECD report on PISA 2003 
and further results will be published in a 
series of OECD thematic reports on PISA 
2003. Data with which researchers can con- 
duct their own analyses are also available at 
ht tp://www.pisa.oecd.org . 
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Appendix A: 
Technical Notes 
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The Program for International Student 
Assessment (PISA) is a system of interna- 
tional assessments that measures 15-year- 
olds’ capabilities in reading literacy, mathe- 
matics literacy, and science literacy every 
three years. PISA was first implemented in 
2000 and is carried out by the Organization 
for Economic Cooperation and Development 
(OECD). In addition to the major subject 
areas, PISA also measures general or 
cross-curricular competencies such as 
learning strategies. In this second cycle, 
PISA 2003, mathematics literacy was the 
major focus, along with the new cross-cur- 
ricular cognitive domain of problem solving. 
This appendix describes features of the 
PISA 2003 survey methodology, including 
sample design, test design, scoring, data 
reliability, and analysis variables. For further 
details about the assessment and any of the 
topics discussed here, see the OECD’s PISA 
2003 Technical Report (Adams forthcoming) 
and the PISA 2000 Technical Report (Adams 
2002 ). 

Sampling, Data Collection, and Response 
Rate Requirements 

To provide valid estimates of student 
achievement and characteristics, the sample 
of PISA students had to be selected in a 
way that represented the full population of 
15-year-old students in each country. The 
international desired population in each 
country consisted of 15-year-olds attending 
both publicly and privately controlled educa- 
tional institutions in grades 7 and higher. A 
minimum of 4,500 students from a minimum 
of 150 schools was required. Within schools, 
a sample of 35 students was to be selected 
in an equal probability sample unless fewer 
than 35 students aged 15 were available (in 
which case all students were selected). 
International standards required that stu- 
dents be sampled based on an age definition 
of 15 years and 3 months to 16 years and 2 
months at the beginning of the testing period. 
The testing period was required not to 



exceed 42 days between March 1, 2003, and 
August 31, 2003. Each country collected its 
own data, following international guidelines 
and specifications. 

A minimum response rate target of 85 per- 
cent was required for initially selected edu- 
cational institutions. In instances in which 
the initial response rate of educational insti- 
tutions was between 65 and 85 percent, an 
acceptable school response rate could still 
be achieved through the use of replacement 
schools. Replacement schools were to be 
selected at the time of sample selection. 

Three school response rate zones — accept- 
able, intermediate, and not acceptable — 
were defined (figure A-1). “Acceptable” 
meant that the country’s data would be 
included in all international comparisons. 
“Not Acceptable” meant that the country’s 
data would be a candidate for not being 
reported in international comparisons 
unless considerable evidence was presented 
that nonresponse bias was minor. “Intermediate” 
meant that a decision on whether or not to 
include the country’s data in comparisons 
would be made while taking into account a 
variety of factors, such as student response 
rates, quality control, closeness of the 
response rates to the acceptable level, etc. 
For the purposes of calculating response 
rates, schools with less than 50 percent of 
students responding were considered non- 
responding and their students were excluded 
from the student response rates. If the stu- 
dent response rates within such schools 
were at least 25 percent, these schools and 
students were included in the PISA 2003 
database. Schools with student response 
rates below 25 percent were not used in any 
type of analysis nor are the data for these 
students or schools available in the PISA 
2003 database. Note that schools with stu- 
dent response rates above 25 percent were 
included in the nonresponse bias analyses 
described in this report. 
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Figure A-1. School response rate requirements for PISA 2003 
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NOTE: A minimum response target of 85 percent was required for initially selected educational institutions. In instances in 
which the initial response rate of educational institutions was between 65 and 85 percent, an acceptable school response rate 
could still be achieved through the use of replacement schools. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 



PISA 2003 also required a minimum partici- 
pation rate of 80 percent of sampled stu- 
dents from original and replacement schools 
within each country. A student was consid- 
ered to be a participant if he or she partici- 
pated in the first testing session or a follow- 
up or makeup testing session. 

Exclusion guidelines allowed for 0.5 percent 
at the school level for approved reasons (for 
example, remote regions or very small 
schools), and 2 percent for special education 
schools. Overall estimated student exclu- 
sions to be under 5 percent. PISA’s intent 
was to be as inclusive as possible. No 
accommodations were offered in the United 
States for PISA. A special one-hour booklet 
with lower difficulty items, which was scaled 
with the regular PISA booklets, was used in 



six countries for schools that would other- 
wise have been excluded. Special booklets 
were used in Austria, Belgium, the Czech 
Republic, Hungary, the Netherlands, and the 
Slovak Republic. Within schools, exclusion 
decisions were made by staff members who 
were knowledgeable about students with 
Individualized Education Programs (lEPs) or 
students who were limited English profi- 
cient, using the following international 
guidelines on possible student exclusions: 

• Functionally disabled students. 

These were students who were perma- 
nently physically disabled in such a way 
that they could not perform in the testing 
situation. Functionally disabled stu- 
dents who could respond were to be 
included in the testing. Any sampled 
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student who was temporarily disabled 
such that s/he could not participate in 
the assessment was considered absent 
from the assessment. 

* Students with mental or emotional 
disabilities. These were students who 
were considered in the professional 
opinion of the school principal or by 
other qualified staff members to be intel- 
lectually disabled or who had been psy- 
chologically tested as such. This includ- 
ed students who were emotionally or 
mentally unable to follow even the gener- 
al instructions of the test. Students were 
not to be excluded solely because of 
poor academic performance or normal 
disciplinary problems. 

* Students with limited proficiency in 
the test language. These were students 
who had received less than one year of 
instruction in the language of the test. 
Generally, these were students who were 
unable to read or speak the language of 
the test (English in the United States) 
and would be unable to overcome the 
language barrier in the test situation. 

Quality monitors from the PISA Consortium 
visited schools in every country to ensure 
testing procedures were carried out in a 
consistent manner across countries. 

Sampling, Data Collection, and Response 
Rates in the United States 

The 2003 PISA school sample was drawn for 
the United States in November 2002. The 
sample design for this school sample was 
developed to retain some of the properties 
of the 2000 PISA U.S. school sample, and to 
follow international requirements as given in 
the PISA sampling manual. Unlike the 2000 
PISA sample, which had a three-stage 
design, the U.S. sample for 2003 was a two- 
stage sampling process with the first stage 



a sample of schools, and the second stage 
a sample of students within schools. For 
PISA in 2000, the U.S. school sample had 
the selection of a sample of geographic 
Primary Sampling Units (PSUs) as the first 
stage of selection. The sample was not clus- 
tered at the geographic level for PISA 2003. 
This change was made in an effort to reduce 
the design effects observed in the 2000 data 
and to spread the respondent burden across 
school districts as much as possible. 

The sample design for PISA was a stratified 
systematic sample, with sampling probabili- 
ties proportional to measures of size. The 
PISA sample had no explicit stratification 
and no oversampling of subgroups. The 
frame was implicitly stratified (i.e., sorted 
for sampling) by five categorical stratifica- 
tion variables: grade span of the school (five 
levels), type of school (public or private), 
region of the country 11 (Northeast, Central, 
West, Southeast), type of location relative to 
populous areas (eight levels), minority sta- 
tus (above or below 15 percent). The last sort 
key within the implicit stratification was by 
estimated enrollment of 15-year-olds based 
on grade enrollments. 

At the same time that the PISA sample was 
selected, replacement schools were identi- 
fied following the PISA guidelines by 
assigning the two schools neighboring the 
sampled school on the frame as replace- 
ments. There were several constraints on 
the assignment of substitutes. One sampled 
school was not allowed to substitute for 
another, and a given school could not be 
assigned to substitute for more than one 
sampled school. Furthermore, substitutes 
were required to be in the same implicit stra- 
tum as the sampled school. If the sampled 
school was the first or last school in the 
stratum, then the second school following or 
preceding the sampled school was identified 
as the substitute. One was designated a first 
replacement and the other a second replace- 



^The Northeast region consists of Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New 
Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont. The Central region consists of Illinois, Indiana, 
Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, Wisconsin, and South Dakota. TheWest region 
consists of Alaska, Arizona, California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oklahoma, Oregon, Texas, Utah, 
Washington, and Wyoming. The Southeast region consists of Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, 
Mississippi, North Carolina, South Carolina, Tennessee, Virginia, West Virginia. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



ment. If an original school refused to partic- 
ipate, the first replacement was then contact- 
ed. If that school also refused to participate, 
the second school was then contacted. 

The U.S. PISA school sample consisted of 
420 schools. This number was increased from 
the international minimum requirement of 
150 to offset school nonresponse, reduce 
design effects, and include additional stu- 
dents in a metric-imperial experiment 
(described below). 

The schools were selected with probability 
proportionate to the school’s estimated 
enrollment of 15-year-olds from the 2003 
NAEP school frame with 2000-2001 school 
data. The data for public schools were from 
the Common Core of Data (CCD), and the 
data for private schools were from the 
Private School Survey (PSS). Any school 
containing at least one 7th- through 12th- 
grade class as of the school year 2000-01 
was included on the school sampling frame. 
Participating schools provided lists of 15- 
year-old students, and a sample of 35 stu- 
dents was selected within each school in an 
equal probability sample. The overall sample 
design for the United States was intended to 
approximate a self-weighting sample of stu- 
dents as much as possible, with each 15- 
year-old student having an equal probability 
of being selected. 

In the United States, for a variety of reasons 
reported by school administrators (such as 
increased testing requirements at the 
national, state, and local levels, concerns 
about timing of the PISA assessment and 
loss of learning time), many schools in the 
original sample declined to participate. As 
it was clear that the United States would not 
meet the minimum response rate standards, 
in order to improve response rates and bet- 
ter accommodate school schedules, a sec- 
ond testing window was opened from 
September to November 2003 with the agree- 
ment of the PISA Consortium. For the fall 



data collection, the school sample included 
only original schools from the sample that 
had refused to participate in the spring but 
indicated a willingness to participate in a 
fall assessment. Substitute schools were 
not included in the fall sample because their 
participation would have had little effect on 
raising the final response rate. In order to 
achieve a comparable sample of students in 
spring and fall, the age definition for stu- 
dents tested in the fall was adjusted such 
that all students tested were the same age. 

Of the 420 sampled schools, 382 were eligible 
(some did not have any 15-year-olds enrolled) 
and 179 agreed to participate in the spring of 
2003. An additional 70 original schools par- 
ticipated in the fall assessment for a total of 
249 participating original schools. The school 
response rate (including spring and fall 
assessments) before replacement was 65 
percent (weighted and unweighted), placing 
the United States in the "intermediate" 
response rate category. The weighted 
school response rate before replacement is 
given by the formula: 

V Un- 
weighted school response _ jty 
rate before replacement ~ 

ie(YUN) 

whereY denotes the set of responding origi- 
nal sample schools with age-eligible stu- 
dents, N denotes the set of eligible non- 
responding original sample schools, Wj 
denotes the base weight for school i, Wj = 
1/Pi , where P, denotes the school selection 
probability for school i, and Ei denotes the 
enrollment size of age-eligible students, as 
indicated on the sampling frame. 

In addition to the 249 participating original 
schools, 13 replacement schools also partici- 
pated in the spring for a total of 262 partici- 
pating schools. 
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A total of 7,598 students were sampled for 
the assessment. Of these students, 261 were 
deemed ineligible because of their enrolled 
grades, birthdays, or other reasons, and 
were removed from the sample. Of the eligi- 
ble 7,337 sampled students, an additional 534 
students were excluded using the criteria 
described above, for a weighted exclusion 
rate of 7 percent. 

Of the 6,803 remaining sampled students, a 
total of 5,456 students participated in the 
assessment in the United States, but 114 of 
these came from schools which had less 
than 50 percent student participation. 

Schools which had less than 50 percent stu- 
dent participation were classified as school 
nonrespondents, and these students (114 
participating students and 187 nonparticipat- 
ing students) were therefore excluded for the 
purposes of calculating student response 
rates.Thus, although data for 5,456 students 
are included in the database, student 
response rates were calculated by subtract- 
ing the 114 students from the 5,456 for a 
total of 5,342 participating students. The 
denominator for the student response rate is 
6,502, which consists of 7,598 sampled stu- 
dents minus the following students: 261 inel- 
igible, 534 excluded, 114 responding students 
from nonresponding schools, and 187 nonre- 
sponding students from nonresponding 
schools. An overall weighted student 
response of 83 percent was achieved (82 per- 
cent unweighted). 

Two separate bias analyses were conducted 
in the United States to address potential 
problems in the data due to school nonre- 
sponse and possible achievement differ- 
ences between students in spring and fall 
testing windows. 



The analysis of school nonresponse was 
conducted in two parts, examining first the 
original sample of schools (spring and fall 
participants) and then the final sample of 
schools (including replacements), treating 
as nonrespondents those schools from 
whom a final response was not received 
(Ferraro, Czuprynski and Williams forthcom- 
ing). Schools with 25 to 49 percent student 
response rates were treated as respondents 
in the nonresponse bias analysis, since their 
data are included in the PISA database. 
Schools with student response rates less 
than 25 percent were treated as nonrespon- 
dents in the analysis and were not included 
in the PISA database. 

In order to compare PISA respondents and 
nonrespondents, it was necessary to match 
the sample of schools back to the sample 
frame to detect as many characteristics as 
possible that might provide information 
about the presence of nonresponse bias. 
Comparing frame characteristics for respon- 
dents and nonrespondents is not always a 
good measure of nonresponse bias if the 
characteristics are unrelated or weakly 
related to more substantive items in the sur- 
vey; however, this was the only approach 
available given that no comparable school or 
student level achievement data were avail- 
able. Frame characteristics were taken from 
the 2000-01 Common Core of Data (CCD) for 
public schools and from the 2000-01 Private 
School Survey (PSS) for private schools. 

For categorical variables, response rates by 
characteristics were calculated. The hypoth- 
esis of independence between the character- 
istics and response status was tested using 
a Rao-Scott modified Chi-square statistic. 
For continuous variables, summary means 
were calculated. The 95 percent confidence 
interval for the difference between the mean 
for respondents and the overall mean was 
tested to see whether or not it included zero. 
In addition to these tests, logistic regression 
models were set up to identify whether any 
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of the frame characteristics were significant 
in predicting response status. All analyses 
were performed using WesVar and replicate 
weights to properly account for the complex 
sample design. The JK2 method was used to 
create the weights. The school base weights 
used in these analyses did not include a 
nonresponse adjustment factor. The base 
weight for each original school was the 
reciprocal of its selection probability. The 
base weight for each replacement school 
was equal to the base weight of the original 
school it replaced. 

Characteristics available for public and pri- 
vate schools included: public/private affilia- 
tion, community type, region, number of age- 
eligible students enrolled, total number of 
students, and percentage of various 
racial/ethnic groups (percentage Asian or 
Pacific Islander; Black, non-Hispanic; 
Hispanic; American Indian or Alaska Native; 
White, non-Hispanic). Percentage of stu- 
dents eligible for free or reduced-price lunch 
was also available for public schools only 
(however, this variable was missing for 50 of 
the 359 public schools). For the original sam- 
ple of schools, two of these variables 
showed a relationship to response status in 
tests of independence and in the multivari- 
ate logistic regression model: region 
(specifically, schools in the Central region 
were less likely to respond) and percentage 
of Asian or Pacific Islander students 
(responding schools had fewer of these stu- 
dents than the original sample schools). 
Using the same analytic procedure for the 
final sample (including replacement 
schools), tests of independence again 
showed that responding schools were more 
likely to be in the West. Responding schools 
were also more likely to have fewer Asian or 
Pacific Islander students and more Black, 
non-Hispanic students. However, the only 
variable found to be significant in the logistic 
regression model predicting response was 
the percentage of Asian or Pacific Islander 
students (again, responding schools were 
likely to have fewer Asian or Pacific Islander 
students). 



While the implications of these analyses for 
the direction of any resulting bias achieve- 
ment are not entirely clear, an attempt was 
made to minimize any bias by incorporating 
the variables in question into the adjustment 
for school nonresponse that was a compo- 
nent of the sampling weights. 

One other country, the United Kingdom, also 
fell below the acceptable range for school 
response rates, although response rate 
problems were largely limited to England 
(Scotland and Northern Ireland also partici- 
pated). In that case, however, the PISA 
Consortium was unable to make adjust- 
ments for any potential bias, and data for the 
United Kingdom are therefore annotated and 
are not included in the main text or figures. 
Data for one additional participating PISA 
2003 country, Brazil, were not available in 
time for production of this report. 

The other U.S. bias analysis aimed to 
address the question of whether there was a 
"session” effect between students tested in 
the spring and fall, in order to provide evi- 
dence for the acceptability of combining 
data from both sessions for the United 
States. Despite PISA’s focus on an age 
sample, concern remained that students 
tested at the beginning of the school year 
might perform worse than their peers tested 
at the end of the previous school year. 

The approach taken was to investigate ses- 
sion effects in a multilevel model, since 
these were school-level effects — all stu- 
dents within a school were in either the 
spring or the fall sessions. Two similar two- 
level models were estimated. In each, stu- 
dent achievement in PISA was modeled as a 
function of various school characteristics (in 
particular those on the sample frame known 
to be related to willingness to participate in 
the original testing window, including pub- 
lic/private status, number of age-eligible 
students, region, and location) and time of 
testing (spring/fall) and, in one model, the 
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student characteristic grade level. In the 
simpler of the student-level models no pre- 
dictors of achievement were included. In the 
second model, student grade level was 
included as a predictor of achievement to 
allow for the possibility that the school 
means predicted in the school-level model 
were affected by differences in the 
spring/fall distribution of students across 
grades. That is, the school-level model was 
predicting mean school achievement adjust- 
ed for grade-level differences. The two mod- 
els proposed were estimated with HLM 
(Raudenbush and Bryk 2002). Neither model 
showed evidence of a statistically significant 
session effect. On this basis, and on the 
basis of the adjustments made to the sam- 
pling weights based on the nonresponse 
bias analysis, the PISA Consortium con- 
cluded that the data for the United States 
was adequate to generalize to the U.S. 15- 
year-old population and should be included 
in the international report and database. 

Table A-1 provides summary information on 
the samples of all countries. A more 
detailed presentation can be found in the 
OECD’s PISA 2003 Technical Report (Adams 
forthcoming). 

Test Development 

The development of the PISA 2003 assess- 
ment instruments was an interactive 
process among the PISA Consortium, vari- 
ous expert committees, and OECD mem- 
bers. The assessment was developed by 
international experts and PISA Consortium 
test developers, and items were reviewed by 
representatives of each country for possible 
bias and relevance to PISA’s goals. The 
intention was to reflect the national, cultur- 
al, and linguistic variety among OECD coun- 
tries. The assessments included material 
selected from among items submitted by 
participating countries as well as items that 
were developed by the Consortium’s test 
developers. 



The final assessment consisted of 85 mathe- 
matics items, 35 science items, 19 problem 
solving items, and 32 reading items allocated 
to 13 test booklets. In the United States, an 
additional 4 test booklets were included in 
PISA 2003 in order to investigate the possi- 
ble effects of the use of metric units on U.S. 
student performance, for a total of 17 book- 
lets (see description that follows). Each 
booklet was made up of 4 test clusters. 

There were 7 mathematics clusters (Mi - 
M 7 ), 2 science clusters (Si - S 2 ), 2 problem 
solving clusters (Pi - P 2 ) and 2 reading clus- 
ters (Ri - R 2 ). The clusters were allocated in 
a rotated design to the 13 booklets. Each 
cluster contained approximately 12 test 
items, equivalent to 30 minutes of test mate- 
rial. Each student took one booklet, with 
about 2 hours worth of testing material. 
Approximately one-third of the mathematics 
literacy items were multiple choice and com- 
plex multiple choice, one-third were closed 
or short response types in which students 
wrote an answer which was simply either 
correct or incorrect, and about one-third 
were open constructed responses for which 
students wrote answers which were marked 
by trained scorers based upon an interna- 
tional scoring guide. In PISA 2003, every 
student answered mathematics items. 
Problem solving, science, and reading items 
were spread throughout other booklets. For 
more information on assessment design, see 
the OECD’s PISA 2003 Technical Report 
(Adams forthcoming). 

In order to examine similarities and differ- 
ences between national and international 
assessments, NCES has sponsored a num- 
ber of comparative studies of assessment 
frameworks and items. In October 2003 a 
study of the NAEP,TIMSS, and PISA 2003 
mathematics assessments was undertaken. 
The aim of the study was to provide informa- 
tion that would be useful in interpreting and 
comparing the results from the three assess- 
ments, based on an in-depth look at the con- 
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Table A-1. Coverage of target population, student and school samples, 
and participation rates in the Program for International 
Student Assessment (PISA), by country: 2003 









Percent 




Country 


Total 


Coverage 


Coverage of 


Overall 


population of 


of 


national 


student 




15-year-olds 


15-year-old 


desired 


exclusion 




(number) 


population 


population 


rate 


OECD countries 


Australia 


268,164 


87.9 


97.8 


2.2 


Austria 


94,515 


90.9 


98.4 


1.6 


Belgium 


120,802 


92.6 


98.5 


1.5 


Canada 


398,865 


82.8 


93.2 


6.8 


Czech Republic 


130,679 


92.7 


98.8 


1.2 


Denmark 


59,156 


87.5 


94.7 


5.3 


Finland 


61,107 


94.7 


96.6 


3.4 


France 


809,053 


90.8 


96.6 


3.4 


Germany 


951,800 


92.9 


98.1 


1.9 


Greece 1 


111,286 


94.5 


96.8 


3.2 


Hungary 


129,138 


82.9 


96.1 


3.9 


Iceland 


4,168 


94.2 


97.4 


2.6 


Ireland 


61 ,535 


89.1 


95.7 


4.3 


Italy 


561 ,304 


85.8 


98.1 


1.9 


Japan 


1,365,471 


90.8 


99.0 


1.0 


Korea, Republic of 


606,722 


87.9 


99.1 


0.9 


Luxembourg 


4,204 


97.1 


98.4 


1.6 


Mexico 


2,192,452 


48.9 


95.7 


4.3 


Netherlands 


194,216 


95.2 


98.1 


1.9 


New Zealand 


55,440 


87.7 


94.9 


5.1 


Norway 


56,060 


94.2 


96.6 


3.4 


Poland 


589,506 


90.7 


96.1 


3.9 


Portugal 


109,149 


88.7 


97.7 


2.3 


Slovak Republic 


84,242 


91.5 


97.0 


3.0 


Spain 


454,064 


75.8 


92.7 


7.3 


Sweden 


109,482 


97.8 


95.8 


4.2 


Switzerland 


83,247 


103.9 


95.6 


4.4 


Turkey 


1,351,492 


35.6 


99.3 


0.7 


United States 


3,979,116 


79.1 


92.7 


7.3 


Non-OECD countries 


Hong Kong-China 


75,000 


96.6 


99.0 


1.0 


Indonesia 2 


4,281,895 


46.0 


99.7 


0.3 


Latvia 


37,544 


89.6 


95.1 


4.9 


Liechtenstein 


402 


84.1 


98.5 


1.5 


Macao-China 


8,318 


78.7 


99.8 


0.2 


Russian Federation 


2,496,216 


86.3 


98.3 


1.7 


Serbia and Montenegro 3 


98,729 


69.5 


94.3 


5.7 


Thailand 


927,070 


68.7 


98.9 


1.1 


Tunisia 4 


164,758 


91.6 


99.6 


0.4 


Uruguay 


53,948 


62.6 


99.6 


0.4 


United Kingdom 5 


768,180 


90.9 


94.6 


5.4 



See notes at end of table. 
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Table A-1. Coverage of target population, student and school samples, and participation 
rates in the Program for International Student Assessment (PISA), by country: 
2003 — Continued 



Percent 



Country 


Weighted 

school 

participation 


Weighted 

school 

participation 


Weighted 

student 

participation 


Number of 
participating 


Number of 




rate before 


rate after 


rate after 


schools after 


participating 




replacement 


replacement 


replacement 


replacement 


students 


OECD countries 


Australia 


86.3 


90.4 


83.3 


314 


12,425 


Austria 


99.3 


99.3 


83.6 


192 


4,597 


Belgium 


83.4 


95.6 


92.5 


282 


8,796 


Canada 


80.0 


84.4 


83.9 


1,066 


27,953 


Czech Republic 


91.4 


99.1 


89.0 


259 


6,320 


Denmark 


84.6 


98.3 


89.9 


205 


4,218 


Finland 


97.4 


100.0 


92.8 


197 


5,796 


France 


88.7 


89.2 


88.1 


163 


4,300 


Germany 


98.1 


98.8 


92.2 


213 


4,600 


Greece 1 


80.6 


95.8 


95.4 


171 


4,627 


Hungary 


97.3 


99.4 


92.9 


252 


4,765 


Iceland 


99.9 


99.9 


85.4 


129 


3,350 


Ireland 


90.2 


92.8 


82.6 


143 


3,880 


Italy 


97.5 


100.0 


92.5 


406 


11,639 


Japan 


87.1 


95.9 


95.1 


144 


4,707 


Korea, Republic of 


95.9 


100.0 


98.8 


149 


5,444 


Luxembourg 


99.9 


99.9 


96.2 


29 


3,923 


Mexico 


94.0 


95.5 


92.3 


1,102 


29,983 


Netherlands 


82.6 


87.9 


88.3 


153 


3,992 


New Zealand 


91.1 


97.6 


85.7 


171 


4,511 


Norway 


87.9 


90.4 


87.9 


180 


4,064 


Poland 


95.1 


98.1 


82.0 


163 


4,383 


Portugal 


99.3 


99.3 


87.9 


152 


4,608 


Slovak Republic 


78.9 


99.1 


91.9 


281 


7,346 


Spain 


98.4 


100.0 


90.6 


383 


10,791 


Sweden 


99.1 


99.1 


92.6 


185 


4,624 


Switzerland 


97.3 


98.5 


94.7 


444 


8,420 


Turkey 


93.3 


100.0 


96.9 


159 


4,855 


United States 


64.9 


68.1 


82.7 


262 


5,456 


Non-OECD countries 


Hong Kong-China 


81.9 


95.9 


90.2 


145 


4,478 


Indonesia 2 


100.0 


100.0 


98.1 


344 


10,761 


Latvia 


95.3 


95.3 


93.9 


157 


4,627 


Liechtenstein 


100.0 


100.0 


98.2 


12 


332 


Macao-China 


100.0 


100.0 


98.0 


39 


1,250 


Russian Federation 


99.5 


100.0 


95.7 


211 


5,974 


Serbia and Montenegro 3 


100.0 


100.0 


91.4 


149 


4,405 


Thailand 


91.5 


100.0 


97.8 


179 


5,236 


Tunisia 4 


100.0 


100.0 


96.3 


149 


4,721 


Uruguay 


93.2 


97.1 


90.8 


239 


5,835 


United Kingdom 5 


64.3 


77.4 


77.9 


361 


9,535 



'Fifteen-year-olds in primary school in Greece were originally excluded from the assessment. Changes in the target population 
definition to 15-year-olds in grades 7 and above required Greece to adjust its data to reflect the fact that 15-year-olds in primary 
school would no longer be considered part of the target population. 

Indonesia excluded 4 provinces and close to 5 percent of its eligible population for security reasons. There were 4,137,103 15-year- 
olds in the total population, but the 4 provinces were already excluded. Therefore, the 144,792 noted as being excluded in these 
provinces was added to this number to get 4,281,895 15-year-olds. The number of enrolled 15-year-olds was noted as 2,968,756 so 
144,792 was also added to this. 

3 Serbia and Montenegro excluded Kosovo; however, there were no estimates for the number of 15-year-olds, so this does not 
appear as an exclusion. 

4 Tunisia noted late in the process that one French school needed to be excluded because of French (rather than Arabic) language. 
The school had 33 eligible students. 

5 Due to low response rates, data for the United Kingdom are not discussed in this report. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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tent of the respective frameworks and items. 
The results showed that PISA used far 
fewer multiple choice items and had a much 
stronger content focus on the "data" area 
(which often deals with using charts and 
graphs), which fits with PISA’s emphasis on 
using materials with a real-world context. 

For more results from the study, see A 
Content Comparison of the NAEP, TIMSS, 
and PISA 2003 Mathematics Assessments 
(Nohara forthcoming). An earlier study 
compared NAEP 2000, PISA 2000, and 
TIMSS 1999 mathematics and science items. 
That study found that PISA items required 
multistep reasoning more often thanTIMSS 
or NAEP and that PISA mathematics and 
science literacy items more often involved 
the interpretation of charts and graphs or 
other "real life” material (Nohara 2001). 

In addition to the cognitive assessment, 
students also received a 30-minute question- 
naire designed to provide information about 
their backgrounds, attitudes, and experiences 
in school. Principals in schools were PISA 
was administered also received a 20-30 
minute questionnaire about their schools. 
Results from the school survey are not dis- 
cussed in this report but are available at 
ht tp://www.pisa.oecd.org . 

Translation and the Metric-Imperial Study 

Source versions of all instruments (assess- 
ment booklets, questionnaires, and manu- 
als) were prepared in English and French 
and translated into the primary language or 
languages of instruction in each nation. 
PISA recommended that countries prepare 
and consolidate independent translations 
from both source versions, and provided 
precise translation guidelines that included 
a description of the features each item was 
measuring and statistical analysis from the 
field trial. In cases where one source lan- 
guage was used, independent translations 
were required and discrepancies reconciled. 
In addition, it was sometimes necessary to 



adapt the instrument for cultural purposes, 
even in nations such as the United States 
that use English as the primary language of 
instruction. For example, words such as 
"lift” might be adapted to “elevator" for the 
United States. The PISA Consortium veri- 
fied the national translation and adaptation 
of all instrumentation. Copies of printed 
materials were sent to the PISA 
Consortium for a final optical check prior to 
data collection. 

As noted, in the United States, an additional 
4 test booklets were included in PISA 2003 
that used adapted versions of 27 mathemat- 
ics items. These items in their original for- 
mat used metric units of measurement, such 
as meters, liters, etc. To investigate the pos- 
sible effects of the use of metric units on 
U.S. student performance, the items were 
adapted to use "imperial” forms with famil- 
iar units such as feet, gallons, and degrees 
Fahrenheit. Differential item analysis 
showed that U.S. students were not disad- 
vantaged by the use of metric units in PISA 
2003. The few discrepancies that were 
observed are possibly due to (1) differences 
in the nature of the two systems (e.g., deci- 
mal vs. duodecimal, or no equivalent wording 
of the units), and (2) difficulties in the modi- 
fication process (e.g., no comparable scor- 
ing guides for some incorrect approaches to 
an item). For more information, see Wilson 
and Xie (2004). 

Test Administration and Quality Assurance 

PISA 2003 emphasized the use of standard- 
ized procedures in all countries. Each coun- 
try collected its own data, based on compre- 
hensive manuals and trainings provided by 
the PISA Consortium to explain the survey’s 
implementation, including precise instruc- 
tions for the work of school coordinators and 
scripts for test administrators for use in 
testing sessions. Test administration in the 
United States was carried out by profes- 
sional staff trained according to the interna- 
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tional guidelines. School staff were asked 
only to assist with listings of students, iden- 
tifying space for testing in the school, and 
specifying any parental consent procedures 
needed for sampled students. Use of calcu- 
lators was at the discretion of participating 
countries; in the United States, this choice 
was left to schools based on school, district, 
or state policy. Students were asked at the 
end of their test booklets if they had used a 
calculator and if so, what type. Approximately 
12 percent of U.S. students did not respond. 
Of the responding students, 91 percent of 
U.S. students reported using a calculator. 
Students who reported using a calculator 
had a mean score of 498 on the combined 
mathematics literacy scale compared to 461 
for those who reported not using a calculator. 

Members of the PISA Consortium visited all 
national centers to review data collection 
procedures, and members of the PISA 
Consortium also visited a randomly selected 
subsample of approximately 10 percent of 
the educational institutions to ensure that 
procedures were being carried out in accor- 
dance with international guidelines. For a 
detailed description of the quality assurance 
procedures, see the OECD’s PISA 2003 
Technical Report (Adams forthcoming). 

Scoring 

At least one-third of the PISA assessment 
was devoted to items requiring constructed 
responses. The process of scoring these 
items was an important step in ensuring the 
quality and comparability of the PISA data. 
Detailed guidelines were developed for the 
scoring guides themselves, training materi- 
als to recruit scorers, and workshop materi- 
als used for the training of national scorers. 
Prior to the national training, the PISA 
Consortium organized training sessions to 
present the material and train the scoring 
coordinators from the participating coun- 
tries, who trained the national scorers. 



For each test item, the scoring guide 
described the intent of the question and how 
to code the students’ responses to each 
item. This description included the credit 
labels — full credit, partial credit, or no cred- 
it — attached to the possible categories of 
response. Also included was a system of 
double-digit coding for some mathematics 
and science items where the first digit repre- 
sented the score, and the second digit repre- 
sented different strategies or approaches 
that students used to solve the problem. The 
second digit generated national profiles of 
student strategies and misconceptions. In 
addition, the scoring guides included real 
examples of students’ responses accompa- 
nied by a rationale for their classification for 
purposes of clarity and illustration. 

To examine the consistency of this marking 
process in more detail within each country 
and to estimate the magnitude of the vari- 
ance components associated with the use of 
markers, the PISA Consortium conducted 
an interscorer reliability study on a subsam- 
ple of assessment booklets. Homogeneity 
analysis was applied to the national sets of 
multiple scoring and compared with the 
results of the field trial. A full description of 
this process and the results can be found in 
the PISA 2003 Technical Report published by 
the OECD (Adams forthcoming). 

Data Entry and Cleaning 

Responsibility for data entry was taken by the 
national project manager from each nation. 
The data collected for PISA 2003 were 
entered into data files with a common interna- 
tional format, as specified in the 
PISA 2003 Data Entry Manual. Data entry was 
facilitated by the use of a common software 
available to all participating nations 
(KeyQuest).The software facilitated the 
checking and correction of data by providing 
various data consistency checks. The data 
were then sent to the Australian Council for 
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Educational Research (ACER) for cleaning. 
ACER’S role in this instance was to check that 
the international data structure was followed, 
check the identification system within and 
between files, correct single case problems 
manually, and apply standard cleaning proce- 
dures to questionnaire files. Results of the 
data cleaning process were documented and 
shared with the national project managers 
and included specific questions when 
required. The national project manager then 
provided ACER with revisions to coding or 
solutions for anomalies. ACER then compiled 
background univariate statistics and prelimi- 
nary classical and Rasch Item Analysis. 
Detailed information on the entire data entry 
and cleaning process can be found in the 
forthcoming PISA 2003 technical report. 

Weighting 

Students included in the final PISA sample 
for a given country were not all equally rep- 
resentative of the full student population, 
even though random samplings of schools 
and students were used to select the sam- 
ple. The use of sampling weights is neces- 
sary for the computation of statistically 
sound, nationally representative estimates. 
Survey weights help adjust for intentional 
over- or under-sampling of certain sectors of 
the population, school or student nonre- 
sponse, or errors in estimating size of a 
school at the time of sampling. Survey 
weighting for PISA 2003 was carried out by 
Westat, as part of the PISA Consortium. 

The internationally defined weighting speci- 
fications for PISA required that each 
assessed student’s sampling weight be the 
product of the inverse of the school’s proba- 
bility of selection, an adjustment for school- 
level nonresponse, the inverse of the stu- 
dent’s probability of selection, and an 
adjustment for student-level nonresponse. 

All PISA analyses were conducted using 
these adjusted sampling weights. The base 



weight for each replacement school was 
equal to the base weight of the original 
school it replaced. 

Scaling and Plausible Values 

PISA used Item ResponseTheory (IRT) 
methods to produce scale scores that summa- 
rized the achievement results. PISA 2003 uti- 
lized a mixed coefficients multinomial logit 
IRT model. This model is similar in principle 
to the more familiar two-parameter IRT 
model. With this method, the performance of 
a sample of students in a subject area or sub 
area can be summarized on a simple scale or 
a series of scales, even when different stu- 
dents are administered different items. 
Because of the reporting requirements for 
PISA and the large number of background 
variables associated with the assessment, 
PISA used these IRT procedures to produce 
accurate results for groups of students while 
limiting the testing burden on individual stu- 
dents. Furthermore, these procedures provid- 
ed data that could be readily used in second- 
ary analyses. IRT scaling provides estimates 
of item parameters (e.g., difficulty, discrimina- 
tion) that define the relationship between the 
item and the underlying variable measured by 
the test. Parameters of the IRT model are 
estimated for each test question, with an 
overall scale being established as well as 
scales for each predefined content area spec- 
ified in the assessment framework. For exam- 
ple, PISA 2003 had five scales describing 
mathematics (a combined score and subscale 
scores in four domains) and one each for 
reading, problem solving, and science. 

The reading literacy and science literacy 
reporting scales used for PISA 2000 and 
PISA 2003 are directly comparable. The 
value of 500, for example, has the same 
meaning as it did in PISA 2000 — that is, the 
mean score in 2000 of the sampled students 
in the 27 OECD countries that participated 
in PISA 2000. 
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This is not the case, however, for mathemat- 
ics literacy. Mathematics literacy, as the 
major domain, was the subject of major 
development work for PISA 2003, and the 
PISA 2003 mathematics literacy assessment 
was much more comprehensive than the 
PISA 2000 mathematics assessment — the 
PISA 2000 assessment covered just two 
{space and shape, and change and relation- 
ships) of the four areas that are covered in 
PISA 2003. Because of this broadening in 
the assessment it was deemed inappropriate 
to report the PISA 2003 mathematics litera- 
cy scores on the same scale as the PISA 
2000 mathematics scores. 

The PISA 2000 and PISA 2003 assessments 
of mathematics, reading and science litera- 
cy are linked assessments. That is, the sets 
of items used to assess each of mathemat- 
ics, reading and science literacy in PISA 
2000 and the sets of items used to assess 
each of mathematics, reading and science 
literacy in PISA 2003 include a subset of 
items common to both sets. For mathemat- 
ics there were 20 items that were used in 
both assessments, in reading there were 28 
items used in both assessments and for sci- 
ence 25 items were used in both assess- 
ments. These common items are referred to 
as link items. 

To establish common reporting metrics for 
PISA 2000 and PISA 2003 the difficulty of 
link items (items used in 2000 and 2003) was 
compared. Items were calibrated using 2003 
data only, and then 2000 items were re-cali- 
brated using the 2003 parameters. Adjustments 
were then made to ability estimate to 
account for booklet effects seen in 2000. The 
comparison of the item difficulties on the 
two occasions was used to determine a 
score transformation that allows the report- 
ing of the data from the two assessments on 
a common scale. The change in the difficulty 
of each of the individual link items is used in 
determining the transformation and as a 



consequence the sample of link items that 
has been chosen will influence the choice of 
transformation. This means that if an alter- 
native set of link items had been chosen the 
resulting transformation would be slightly 
different. The consequence is an uncertainty 
in the transformation due to the sampling of 
the link items, just as there is an uncertainty 
in values such as country means due to the 
use of a sample of students. The section on 
statistical testing below describes how this 
uncertainty has been accounted for in mak- 
ing comparisons over time. 

Plausible Values 

During the scaling phase, plausible values 
were used to characterize scale scores for 
students participating in the assessment. To 
keep student burden to a minimum, PISA 
administered few assessment items to each 
student — too few to produce accurate con- 
tent-related scale scores for each student. 

To account for this, PISA generated five 
possible scale scores for each student that 
represented selections from the distribution 
of scale scores of students with similar 
backgrounds who answered the assessment 
items the same way. The plausible values 
technology is one way to ensure that the 
estimates of the average performance of 
student populations and the estimates of 
variability in those estimates are more accu- 
rate than those determined through tradi- 
tional procedures, which estimate a single 
score for each student. During the construc- 
tion of plausible values, careful quality con- 
trol steps ensured that the subpopulation 
estimates based on these plausible values 
were accurate. 

It is important to recognize that plausible 
values are not test scores for individuals and 
they should not be treated as such. 

Plausible values are randomly drawn from 
the distribution of scores that could be rea- 
sonably assigned to each individual. As 
such, the plausible values contain random 
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error variance components and are not opti- 
mal as scores for individuals. The PISA stu- 
dent file contains many plausible values, five 
for each of the PISA 2003 cognitive scales 
(combined mathematics literacy scale, four 
mathematics literacy subscales, reading lit- 
eracy, science literacy, and problem solving). 
If an analysis is to be undertaken with one of 
these cognitive scales, then (ideally) the 
analysis should be undertaken five times, 
once with each of the five relevant plausible 
value variables. The results of these five 
analyses are averaged and then significance 
tests that adjust for variation between the 
five sets of results are computed. 

PISA uses the plausible value methodology 
to represent what the true performance of an 
individual might have been, had it been 
observed, using a small number of random 
draws from an empirically derived distribu- 
tion of score values based on the student’s 
observed responses to assessment items 
and on background variables. Each random 
draw from the distribution is considered a 
representative value from the distribution of 
potential scale scores for all students in the 
sample who have similar characteristics and 
identical patterns of item responses. The 
draws from the distribution are different 
from one another to quantify the degree of 
precision (the width of the spread) in the 
underlying distribution of possible scale 
scores that could have caused the observed 
performance. The PISA plausible values 
function like point estimates of scale scores 
for many purposes, but they are unlike true 
point estimates in several respects. They 
differ from one another for any particular 
student, and the amount of difference quan- 
tifies the spread in the underlying distribu- 
tion of possible scale scores for that stu- 
dent. Because of the plausible values 
approach, secondary researchers can use 
the PISA data to carry out a wide range of 
analyses. 



Levels 

While the basic form of measurement in 
PISA describes student literacy in each 
country in terms of a range of scale scores, 
PISA also treats proficiency in mathematics 
literacy in terms of six described levels, and 
proficiency in problem solving in three 
described levels. In both cases, increasing 
levels represent tasks of increasing com- 
plexity. As a result, the findings are reported 
in terms of percentages of the population 
proficient at handling tasks of different lev- 
els of difficulty. 

Each of the five mathematics literacy 
scales — the combined score and the four 
subscale scores — is divided into six levels 
based on the type of knowledge and skills 
students need to demonstrate at a particular 
level. A seventh level (below level 1) is made 
up of students whose abilities could not be 
accurately described based on their 
responses. Exact cut point scores are as fol- 
lows: below level 1 (a score less than or 
equal to 357.77); level 1 (a score greater than 
357.77 and less than or equal to 420.07); level 
2 (a score greater than 420.07 and less than 
or equal to 482.38); level 3 (a score greater 
than 482.38 and less than or equal to 544.68); 
level 4 (a score greater than 544.68 and less 
than or equal to 606.99); level 5 (a score 
greater than 606.99 and less than or equal to 
669.30); level 6 (a score greater than 669.30. 
The tasks that represent each level of per- 
formance for the specific mathematics 
processes on the combined mathematics lit- 
eracy scale are described in exhibit 5. Exhibit 
A-1 describes the kind of tasks that repre- 
sent each level of performance on the math- 
ematics subscales. 

The problem-solving scale is divided into 
three levels based on the type of knowledge 
and skills students must demonstrate at a 
particular level. A fourth level (below level 1) 
is made up of students whose abilities could 
not be accurately described based on their 
responses. In order to reach a particular 
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Exhibit A-1. Description of proficiency levels for mathematics literacy 
subscales: 2003 





Task descriptions 


Proficiency level 


Space and shape 


Quantity 


Level 1 


Students at Level 1 or 2 can 
work with a single mathemati- 
cal representation where the 
mathematical content is direct 
and clearly presented, use 
mathematical thinking in famil- 
iar contexts, identify geometric 
patterns, and apply basic geo- 
metric concepts. 


Students at Level 1 or 2 can 
interpret simple tables, carry 
out basic arithmetic calcula- 
tions, work with simple quanti- 
tative models, interpret a sim- 
ple quantitative model (e.g., a 
proportional relationship), and 
apply the model using basic 
arithmetic calculations. 


Level 2 


Level 3 


Students at Level 3 can begin 
to use visual and spatial rea- 
soning, begin linking different 
representations, use elemen- 
tary problem solving (devising 
simple strategies), apply sim- 
ple algorithms, and interpret 
textual descriptions of unfamil- 
iar geometric situations. 


Students at Level 3 can use sim- 
ple problem solving strategies, 
interpret tables to locate infor- 
mation, carry out well-described 
calculations, interpret a text 
description of a sequential calcu- 
lation process, correctly imple- 
ment the process, and use basic 
problem-solving procedures. 


Level 4 


Students at Level 4 can use 
more advanced and flexible 
reasoning, link and integrate 
different representations, use 
multi-step processes, use 
well-developed spatial visuali- 
zation and interpretation, and 
use reasoning about numeric 
relationships in geometric 
problems. 


Students at Level 4 can work 
effectively with simple models 
of complex situations, use rea- 
soning skills, insight and inter- 
pretation with different repre- 
sentations, use a variety of cal- 
culation skills to solve prob- 
lems, and accurately apply a 
given numeric algorithm involv- 
ing a number. 


Level 5 


Students at Level 5 have the 
ability to make or work with 
assumptions, use insight, inter- 
pretation and linking of differ- 
ent representations, and can 
carry out multiple and sequen- 
tial processes. They can also 
use well-developed spatial rea- 
soning. 


Students at Level 5 have the 
ability to work effectively with 
increasingly complex situa- 
tions and models and have 
well-developed reasoning 
skills. They can also use insight 
and interpretation of different 
representations and carry out 
multiple sequential problems. 


Level 6 


Students at Level 6 can manipu- 
late complex and multiple repre- 
sentations, link different informa- 
tion, use significant insight and 
reflection, make generalizations, 
communicate the solution and 
explanation of a problem in 
unstructured form, and interpret 
complex textual descriptions and 
relate these to other problems. 


Students at Level 6 can con- 
ceptualize and work with com- 
plex mathematical processes 
and relationships, use 
advanced thinking and reason- 
ing skills to link multiple con- 
texts, use sequential calcula- 
tion processes, and conceptu- 
alize complex mathematical 
processes. 



See notes at end of exhibit. 
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Exhibit A-1. Description of proficiency levels for mathematics literacy 
subscales: 2003 — Continued 





Task descriptions 


Proficiency level 


Change and relationships 


Uncertainty 


Level 1 


Students at Level 1 or 2 can 
work with simple algorithms, 
formula, and procedures, link 
text with single representa- 
tions, begin to interpret and 
use elementary reasoning, and 
interpret text to produce a sim- 
ple mathematical model in an 
applied context. 


Students at Level 1 or 2 can 
understand and use basic 
probabilistic ideas in familiar 
experimental contexts, locate 
statistical information present- 
ed in familiar graphical form, 
and understand basic probabil- 
ity concepts in the context of a 
simple and familiar experiment. 


Level 2 


Level 3 


Students at Level 3 can work 
with related representations 
(text, graph, table and simple 
algebra) including some inter- 
pretation and reasoning, inter- 
pret unfamiliar graphical repre- 
sentations of real-world situa- 
tions, and link and connect mul- 
tiple related representations. 


Students at Level 3 can inter- 
pret information and data, link 
different information sources, 
use basic reasoning with sim- 
ple probability concepts, inter- 
pret tabular information, use 
insight into aspects of data 
presentation, and link data to 
suitable chart types. 


Level 4 


Students at Level 4 can under- 
stand and work with multiple 
representations, including 
explicitly mathematical mod- 
els of real-world situations, 
carry out a seguence of calcu- 
lations involving percentage or 
proportion, and show insight 
into three-dimensional geo- 
metric problems. 


Students at Level 4 can use 
basic statistical and probability 
concepts combined with logical 
reasoning in less familiar con- 
texts, use argumentation based 
on interpretation of data, inter- 
pret text, including in an unfa- 
miliar (scientific) context, and 
translate text description into 
mathematics problems. 


Level 5 


Students at Level 5 have quite 
advanced use of algebraic and 
other formal mathematical 
expressions and models and 
have the ability to link formal 
mathematical representations 
to complex real-world situa- 
tions. They can also solve com- 
plex and multi-step problems. 


Students at Level 5 can apply 
statistical knowledge in situa- 
tions that are somewhat struc- 
tured and where the mathemat- 
ical representation is partially 
apparent and use reasoning 
and insight to interpret given 
information. 


Level 6 


Students at Level 6 can use sig- 
nificant insight, well-developed 
reasoning skills and explicit 
technical knowledge to solve 
problems and to begin to gener- 
alize mathematical solutions to 
complex real-world problems 
and can interpret complex 
mathematical information in 
the context of a problem. 


Students at Level 6 can use 
high-level thinking and reason- 
ing skills in statistical or prob- 
abilistic contexts to create 
mathematical representations 
of real-world situations, use 
insight, reflection and argu- 
mentation to communicate 
arguments and explanations, 
and interpret and reflect. 



NOTE: In order to reach a particular proficiency level, a student must have been able to correctly 
answer a majority of items at that level. Students were classified into mathematics literacy lev- 
els according to their scores. Exact cut point scores are as follows: below level 1 (a score less 
than or equal to 357.77 ); level 1 (a score greater than 357.77 and less than or equal to 420.07); level 2 
(a score greater than 420.07 and less than or equal to 482.38)', level 3 (a score greater than 482.38 and 
less than or equal to 544.68)', level 4 (a score greater than 544.68 and less than or equal to 606.99); 
level 5 (a score greater than 606.99 and less than or equal to 669.3); level 6 (a score greater than 
669.3). 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for 
International Student Assessment (PISA), 2003. 
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proficiency level, a student must have been 
able to correctly answer a majority of items 
at that level. Students were classified into 
problem-solving levels according to their 
scores. Exact cut point scores are as fol- 
lows: below level 1 (a score less than or 
equal to 404.06); level 1 (a score greater than 
404.06 and less than or equal to 498.08); level 
2 (a score greater than 498.08 and less than 
or equal to 592.10); level 3 (a score greater 
than 592.10). 

All students within a level are expected to 
answer at least half of the items from that 
level correctly. Students at the bottom of a 
level have a 62 percent chance of success on 
the easiest items from that level and a 42 
percent chance of success on the hardest 
items from that level (overall response prob- 
ability was 62). Students at the top of a level 
are able to provide the correct answers to 
about 70 percent of all items from that level, 
have a 62 percent chance of success on the 
hardest items from that level, and have a 78 
percent chance of success on the easiest 
items from that level. Students just below 
the top of a level would score less than 50 
percent on an assessment of the next higher 
level. Students at a particular level not only 
demonstrate the knowledge and skills asso- 
ciated with that level but also the proficien- 
cies defined by lower levels. Thus, all stu- 
dents proficient at level 3 are also proficient 
at levels 1 and 2. Patterns of responses for 
students below level 1 suggest they are 
unable to answer at least half of the items in 
level 1 correctly. 

Data Limitations 

As with any study, there are limitations to 
PISA 2003 that researchers should take into 
consideration. Estimates produced using 
data from PISA 2003 are subject to two 
types of error, nonsampling and sampling 
errors. Nonsampling errors can be due to 
errors made in the collection and processing 
of data. Sampling errors can occur because 
the data were collected from a sample rather 
than a complete census of the population. 



Nonsampling Errors 

Nonsampling error is a term used to 
describe variations in the estimates that 
may be caused by population coverage limi- 
tations, nonresponse bias, and measurement 
error, as well as data collection, processing, 
and reporting procedures. For example, the 
sampling frame was limited to regular public 
and private schools in the 50 states and the 
District of Columbia. The sources of non- 
sampling errors are typically problems like 
unit and item nonresponse, the differences 
in respondents’ interpretations of the mean- 
ing of the questions, response differences 
related to the particular time the survey was 
conducted, and mistakes in data prepara- 
tion. Some of these issues (particularly unit 
nonresponse) are discussed above in the 
section on U.S. sampling and data collec- 
tion. 

Missing Data 

There are four kinds of missing data. 
“Nonresponse” data occurs when a respon- 
dent was expected to answer an item but no 
response is given. Responses that are 
“missing or invalid” occur in multiple-choice 
items where an invalid response is given. 

The code is not used for open-ended ques- 
tions. An item is “not applicable” when it is 
not possible for the respondent to answer 
the question. Finally, items that are “not 
reached” are consecutive missing values 
starting from the end of each test session. 
All four kinds of missing data are coded 
differently in the PISA 2003 database. 

Missing background data are not included in 
the analyses for this report and are not 
imputed. In general, item response rates for 
variables discussed in this report were over 
the NCES standard of 85 percent to report 
without notation (table A-2). The one case 
in which more than 15 percent of the student 
responses were missing (for New Zealand 
for student report of parent occupation, with 
an item response rate of 84 percent) is 
flagged in the supporting statistical data 
tables in appendix B. 
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In general, it is difficult to identify and esti- 
mate either the amount of nonsampling error 
or the bias caused by this error. In PISA 
2003, efforts were made to prevent such 
errors from occurring and to compensate for 
them when possible. For example, the design 
phase entailed a field test that evaluated 
items as well as the implementation proce- 
dures for the survey. It should also be rec- 
ognized that most background information 
was obtained from students’ self-reports, 
which are subject to respondent bias. One 
potential source of respondent bias in this 
survey was social desirability bias, for exam- 
ple, if students reported that they were good 
at mathematics. 

Sampling Errors 

Sampling errors occur when the discrepancy 
between a population characteristic and the 
sample estimate arises because not all 
members of the reference population are 
sampled for the survey. The size of the sam- 
ple relative to the population and the vari- 
ability of the population characteristics both 
influence the magnitude of sampling error. 
The particular sample of 15-year-old stu- 
dents from the 2002-03 school year was just 
one of many possible samples that could 
have been selected. Therefore, estimates 
produced from the PISA 2003 sample may 
differ from estimates that would have been 
produced had another sample of 15-year-old 
students been drawn. This type of variability 
was called sampling error because it arises 
from using a sample of 15-year-old students 
in 2002, rather than all 15-year-old students 
in that year. 

The standard error is a measure of the vari- 
ability due to sampling when estimating a 
statistic. The approach used for calculating 
sampling variances in PISA was the 
Balanced Repeated Replication (BRR), or 
Balanced Half-Samples (Fay’s method). 
Standard errors can be used as a measure 
for the precision expected from a particular 
sample. 



Standard errors for all of the estimates are 
included in appendix B to this report. These 
standard errors can be used to produce con- 
fidence intervals.There is a 95 percent 
chance that the true average lies within the 
range of 1.96 times the standard errors above 
or below the estimated score. For example, it 
was estimated that 15.5 percent of U.S. stu- 
dents scored at level 1 on the combined 
mathematics literacy scale, and this statis- 
tic had a standard error of 0.81. Therefore, it 
can be stated with 95 percent confidence 
that the actual percentage of U.S. students 
at level 1 for the total population in 2003 was 
between 13.9 and 17.1 percent (1.96 x 0.81 = 
1.59; confidence interval = 15.5 +/- 1.59). 

Descriptions of Background Variables 

Full PISA 2003 student and school question- 
naires are available at 
ht tp://nces.ed.gov/survevs/pisa or 
ht tp://www.pisa.oecd.org . 

Socioeconomic Status 

The measure of student socioeconomic status 
used in PISA 2003 is based on the occupa- 
tional status of the student’s father and/or 
mother (whichever is higher) as reported by 
the student. Parental occupation was coded 
to 4 digits based on the International 
Standard Classification of Occupations 
(ISCO). Occupational codes were in turn 
mapped onto an internationally comparable 
index of occupational status, the International 
Socioeconomic Index (ISEI), developed by 
Ganzeboom, De Graaf, andTreiman (1992). 
Using the index, students were assigned num- 
bers ranging from about 16 to 90 based on 
their parents’ occupations, so that they were 
arrayed on a continuum from low to high 
socioeconomic status, rather than placed into 
discrete categories. The range of ISEI scores 
given for the 1988 ISCO occupations listed in 
Ganzeboom andTreiman (1996) goes from 16, 
the lowest (agricultural laborer), to 90, the 
highest (judge). Typical occupations among 
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Table A-2. Response rates of 15-year-old students for selected background 
variables, by country: 2003 







Selected background variable 




Country 


Sex 


Socioeconomic 
status (ISEI) 


Race/ Type of mathe- 
ethnicity matics class 


OECD countries 


Australia 


100 


95 


t 


t 


Austria 


100 


96 


t 


t 


Belgium 


100 


94 


t 


t 


Canada 


97 


93 


t 


t 


Czech Republic 


100 


96 


t 


t 


Denmark 


100 


97 


t 


t 


Finland 


100 


99 


t 


t 


France 


100 


96 


t 


t 


Germany 


99 


91 


t 


t 


Greece 


100 


94 


t 


t 


Hungary 


100 


94 


t 


t 


Iceland 


100 


98 


t 


t 


Ireland 


100 


96 


t 


t 


Italy 


100 


98 


t 


t 


Japan 


100 


89 


t 


t 


Korea, Republic of 


100 


98 


t 


t 


Luxembourg 


100 


96 


t 


t 


Mexico 


100 


95 


t 


t 


Netherlands 


100 


93 


t 


t 


New Zealand 


100 


84 


t 


t 


Norway 


100 


97 


t 


t 


Poland 


100 


98 


t 


t 


Portugal 


100 


97 


t 


t 


Slovak Republic 


100 


96 


t 


t 


Spain 


100 


96 


t 


t 


Sweden 


100 


97 


t 


t 


Switzerland 


100 


97 


t 


t 


Turkey 


100 


88 


t 


t 


United States 


100 


94 


97 


95 


Non-OECD countries 


Hong Kong-China 


100 


96 


t 


t 


Indonesia 


100 


91 


t 


t 


Latvia 


100 


97 


t 


t 


Liechtenstein 


100 


97 


t 


t 


Macao-China 


100 


97 


t 


t 


Russian Federation 


100 


98 


t 


t 


Serbia and Montenegro 


100 


94 


t 


t 


Thailand 


100 


95 


t 


t 


Tunisia 


100 


95 


t 


t 


Uruguay 


100 


90 


t 


t 


United Kingdom 1 


100 


94 


t 


t 



f Not applicable. 

'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: Cases where more than 15 percent of the student responses are missing are flagged in the support- 
ing statistical data tables in appendix B. For more information about the variables, see the Description of 
Variables section in appendix B.The overall percentage refers to the sample estimate for the overall 15- 
year-old student population. The International Socioeconomic Index (ISEI) is an internationally compara- 
ble index of occupational status, with a range of approximately 16 to 90, developed by Ganzeboom, De 
Graaf, andTreiman (1992). 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



63 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



parents of 15-year-olds with between 16 and 
35 points on the ISEI scale include small- 
scale farmer, metalworker, mechanic, taxi or 
truck driver, and waiter/waitress. Between 35 
and 53 index points, the most common occu- 
pations are bookkeeping, sales, small busi- 
ness management, and nursing. As the 
required skills increase, so does the status of 
the occupation. Between 54 and 70 points, 
typical occupations are marketing manage- 
ment, teaching, civil engineering, and account- 
ant. Finally, between 71 and 90 points, the top 
international quarter of the index, occupations 
include medicine, university teaching, and law 
(OECD 2001). 

Race/Ethnicity 

In the United States, students’ race/ethnici- 
ty was obtained through student responses 
to a two-part question. Students were asked 
first whether they were Hispanic or Latino, 
and then asked whether they were members 
of the following racial groups: American 
Indian or Alaska Native, Asian, Black or 
African American, Native Hawaiian or other 
Pacific Islander, orWhite. Multiple respons- 
es to the race classification question were 
allowed. Results are shown separately for 
Asians, Blacks, Hispanics, Whites, and stu- 
dents who selected more than one race. 
Students identifying themselves as Hispanic 
and also other races were included in the 
Hispanic group, rather than in a racial group. 

Confidentiality and Disclosure Limitations 

The PISA 2003 data are hierarchical and 
include school data and student data from 
the participating schools. Confidentiality 
analyses for the United States were 
designed to provide reasonable assurance 
that public use data files issued by the PISA 
Consortium would not allow identification of 
individual U.S. schools or students when 
compared against public data collections. 
Disclosure limitation included the identifica- 
tion and masking of potential disclosure-risk 



PISA schools and adding an additional 
measure of uncertainty of school and stu- 
dent identification through random swapping 
of data elements within the student and 
school files. 

Statistical Procedures 

Tests of Significance 

Comparisons made in the text of this report 
have been tested for statistical significance. 
For example, in the commonly made compar- 
ison of country averages against the average 
of the United States, tests of statistical sig- 
nificance were used to establish whether or 
not the observed differences from the U.S. 
average were statistically significant. 

The estimation of the standard errors that 
are required in order to undertake the tests 
of significance is complicated by the com- 
plex sample and assessment designs which 
both generate error variance. Together they 
mandate a set of statistically complex proce- 
dures in order to estimate the correct stan- 
dard errors. As a consequence, the estimat- 
ed standard errors contain a sampling vari- 
ance component estimated by Balanced 
Repeated Replication (BRR) — the Fay 
method of BRR; and, where the assessments 
are concerned, there is an additional imputa- 
tion variance component arising from the 
assessment design. Details on the BRR 
procedures used can be found in the WesVar 
4.0 User’s Guide (Westat 2000). 

In almost all instances, the tests for signifi- 
cance used were standard t tests. These fell 
into two categories according to the nature 
of the comparison being made: comparisons 
of independent and non-independent sam- 
ples. In PISA, country samples are inde- 
pendent. To determine whether the average 
scores for two countries are different we 
test the null hypothesis: 

Ho'.p (country 1)~ [J (country2 ) — 0 
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To test this hypothesis, the two observed 
values and their respective standard errors 
are needed to perform a t test.The standard 
error on the estimate for some statistic 0 is: 

« . . = /« 2 + « 2 . 

(£,Y£,) ^ (0 (Jj) 

Thus, in simple comparisons of independent 
averages, such as the average score of 
country 1 with that of country 2, the following 
formula was used to compute the t-statistic: 

t = esti - est 2 / SQRT[(sei) 2 + (se 2 ) 2 ] 

where est, and est 2 are the estimates being 
compared (e.g., averages of country 1 and 
country 2) and sei and se 2 are the correspon- 
ding standard errors of these averages. 

This test may also be used for comparisons 
within a particular country if the categorical 
variable used to define the groups being 
compared was used as an explicit stratifica- 
tion variable; however, there was no explicit 
stratification used in the United States 
sample. 

The second type of comparison used in this 
report occurred when comparing differences 
of non-subset, non-independent groups. 
When this occurs, the correlation and relat- 
ed covariance between the groups must be 
taken into account, such as when comparing 
a country mean with the OECD mean which 
includes that particular country, or when 
comparing the average scores of males ver- 
sus females within the United States. 

How are scores like those for P(boys) and P(giris) 
correlated? Suppose that in the school sam- 
ple, a coeducational school attended by low 
achievers is replaced by a coeducational 
school attended by high achievers. The coun- 
try mean will increase slightly, as well as 
the males’ and the females’ means. If such 
a school replacement process is continued, 
P(boys) and P(giris) will likely increase in a similar 



pattern. Indeed, a coeducational school 
attended by high achieving males is usually 
also attended by high achieving females. 
Therefore, the covariance between Ptboys) and 
P (giris) will be positive. 

What does the covariance between the two 
variables, i.e., Ptboys ;, P (giris), tell us? A positive 
covariance means that if Ptboys) increases 
then P (giris) will also increase. A covariance 
equal or close to 0 means that P(boys) can 
increase or decrease with P (giris) remaining 
unchanged. Finally, a negative covariance 
means that if P(boys) increases, then P (giris) will 
decrease, and inversely. 

Next, to determine whether the females' 
performance differs from the males’ per- 
formance, for example, as for all statistical 
analyses, a null hypothesis has to be tested. 
In this particular example, it will consist of 
computing the difference between the 
males’ performance mean and the females’ 
performance mean (or the inverse). The null 
hypothesis will be: 

H 0 ■ P (boys) P (girls) — 0 . 

The variance of the observed difference is 
needed to test this null hypothesis. The vari- 
ance of a difference is equal to the sum of 
the variances of the two initial variables 
minus two times the covariance between the 
two initial variables. A sampling distribution 
has the same characteristics as any distri- 
bution, except that units consist of sample 
estimates and not observations. Therefore, 
the sampling variance of a difference is 
equal to the sum of the two initial sampling 
variances minus two times the covariance 
between the two sampling distributions on 
the estimates. 

v 2 (P x -P y ) = v 2 (p x )+ v 2 (P Y )- 2co v(p Xi p Y ) 

The estimation of the covariance between, 
for instance, P(bo yS ) and P (giris) requires the 
selection of several samples and then the 
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analysis of the variation of P(bo yS ) in conjunc- 
tion with P (gMs). Such a procedure is, of 
course, unrealistic. Therefore, as for any com- 
putation of a standard error in PISA, replica- 
tion methods using the supplied replicate 
weights are used to estimate the standard 
error on a difference. Use of the replicate 
weights implicitly incorporates the covari- 
ance between the two estimates into the esti- 
mate of the standard error on the difference. 

To test such comparisons, the following for- 
mula was used to compute the t statistic: 

t = estgrpi - estg r p2 /se(estg r pi - estg r p2) 

Estgrpi and est grp 2 are the non-independent 
groups estimates being compared; 
se (estgrpi - est grp 2) is the standard error of 
the difference calculated using Balanced 
Repeated Replication (BRR) to account for 
any covariance between the estimates for 
the two non-independent groups. 

A third type of comparison (addition of a stan- 
dard error term to the standard t test shown 
above for simple comparisons of independent 
averages) was also used when analyzing 
change in performance over time. The uncer- 
tainty that results from the link item sampling 
(described in the scaling section above) is 
referred to as linking error and this error must 
be taken into account when making certain 
comparisons between PISA 2000 and PISA 
2003 results. Just as with the error that is 
introduced through the process of sampling 
students, the exact magnitude of this linking 
error cannot be determined. We can, however, 
estimate the likely range of magnitudes for 
this error and take this error into account when 
interpreting PISA results. As with sampling 
errors, the likely range of magnitude for the 
errors is represented as a standard error. The 
standard error of linking for reading is 3.74, the 
standard error of linking for science is 3.02, 
and the standard error for mathematics ( space 
and shape scale) is 6.01 and mathematics 
(change and relationships scale) is 4.84. 



In PISA, in each of the three subject matter 
areas, a common transformation was esti- 
mated from the link items, and this transfor- 
mation was applied to all participating coun- 
tries. It follows that any uncertainty that 
was introduced through the linking is com- 
mon to all students and all countries. Thus, 
for example, suppose the unknown linking 
error (between PISA 2000 and PISA 2003) in 
reading literacy resulted in an over-estima- 
tion of student scores by two points on the 
PISA 2000 scale. It follows that every stu- 
dent’s score will be over-estimated by two 
score points. This over-estimation will have 
effects on certain, but not all, summary sta- 
tistics computed from the PISA 2003 data. 
For example, consider the following: 

• each country’s mean will be over-esti- 
mated by an amount equal to the link 
error, (in our example this is two score 
points); 

• the mean performance of any subgroup 
will be over-estimated by an amount 
equal to the link error (in our example 
this is two score points); 

• the standard deviation of student scores 
will not be affected because the over- 
estimation of each student by a common 
error does not change the standard devi- 
ation; 

• the difference between the mean scores 
of two countries in PISA 2003 will not be 
influenced because the over-estimation 
of each student by a common error will 
have distorted each country’s mean by 
the same amount; 

• the difference between the mean scores 
of two groups (e.g., males and females) 
in PISA 2003 will not be influenced, 
because the over-estimation of each stu- 
dent by a common error will have distort- 
ed each group’s mean by the same 
amount; 
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• the difference between the performance 
of a group of students (e.g., a country) 
between PISA 2000 and PISA 2003 will 
be influenced because each student’s 
score in PISA 2003 will be influenced by 
the error; and 

• a change in the difference in perform- 
ance between two groups from PISA 
2000 to PISA 2003 will not be influenced. 
This is because neither of the compo- 
nents of this comparison, which are dif- 
ferences in scores in 2000 and 2003 
respectively, is influenced by a common 
error that is added to all student scores 
in PISA 2003. 

In general terms, the linking error need only 
be considered when comparisons are being 
made between PISA 2000 and PISA 2003 
results, and then usually only when group 
means are being compared. Because the 
linking error need only be used in a limited 
range of situations we have chosen not to 
report the linking error in the tables included 
in this report. The general formula is given by: 

t=est-\ -est 2 / SQRT[(sei) 2 + (se 2 ) 2 +(sen n king) 2 ] 

The most obvious example of a situation 
where there is a need to use linking error is 
in the comparison of the mean performance 
for a country between PISA 2000 and PISA 
2003. For example, let us consider a compar- 
ison between 2000 and 2003 of the perform- 
ance of Italy in reading. The mean perform- 
ance of Italy in 2000 was 487 with a standard 
error of 2.9, while in 2003 the mean was 476 
with a standard error of 3.0. The standard- 
ized difference in the Italian mean is 1.97, 
which is computed as follows: 



1.97 = (487 - 476) /j2.9 2 + 3.0 2 + 3.7 2 



In the U.S. report on PISA 2000, a Bonferroni 
adjustment was used in all multiple compar- 
isons of countries. This was not the case in 
2003, which may result in some differences 
in how 2000 results are reported in 2003. This 
may also result in some differences between 
the PISA 2003 U.S. and OECD reports 
(which uses a Bonferroni adjustment for 
multiple comparisons of country averages). 
The discontinuation of the use of the 
Bonferroni adjustment for multiple compar- 
isons was made in order to avoid the possi- 
bility that comparisons of achievement 
between countries could be interpreted dif- 
ferently depending on the numbers of coun- 
tries compared. 






and is statistically significant. 
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Table B-1. Percentage distribution of 15-year-old students, by grade and country: 2003 



Country 


7 ,h 




8 th 




9 th 




10 th 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


0.5 


0.04 


4.7 


0.11 


35.9 


0.25 


52.7 


0.26 


OECD countries 


Australia 


# 


t 


t 


t 


8.3 


0.40 


72.3 


0.70 


Austria 


t 


t 


5.1 


0.91 


43.1 


1.47 


51.3 


1.55 


Belgium 


t 


t 


3.7 


0.34 


29.5 


0.67 


65.2 


0.64 


Canada 


0.6 


0.15 


2.5 


0.26 


13.7 


0.46 


82.0 


0.58 


Czech Republic 


t 


t 


2.8 


0.34 


44.7 


1.14 


52.4 


1.14 


Denmark 


t 


t 


9.1 


0.58 


86.9 


0.85 


3.8 


0.65 


Finland 


t 


t 


12.4 


0.51 


87.3 


0.51 


# 


t 


France 


t 


t 


5.4 


0.57 


34.9 


1.17 


57.3 


1.11 


Germany 


1.7 


0.27 


15.0 


0.79 


59.8 


0.74 


23.2 


0.57 


Greece 


t 


t 


2.1 


0.37 


6.6 


0.99 


76.1 


1.41 


Hungary 


1.1 


0.21 


5.0 


0.47 


65.1 


0.71 


28.8 


0.62 


Iceland 


# 


t 


# 


t 


# 


t 


100.0 


t 


Ireland 


# 


t 


2.8 


0.32 


60.9 


1.31 


16.7 


1.35 


Italy 


t 


t 


1.4 


0.36 


14.2 


0.60 


80.0 


0.82 


Japan 


# 


t 


# 


t 


# 


t 


100.0 


t 


Korea, Republic of 


# 


t 


# 


t 


1.6 


0.23 


98.3 


0.23 


Luxembourg 


# 


t 


14.9 


0.22 


55.8 


0.25 


29.3 


0.18 


Mexico 


3.6 


0.49 


11.0 


0.96 


40.8 


2.38 


43.7 


2.81 


Netherlands 


t 


t 


4.4 


0.56 


45.6 


1.14 


49.3 


1.31 


New Zealand 


# 


t 


t 


t 


6.8 


0.48 


89.2 


0.51 


Norway 


# 


t 


# 


t 


* 


t 


98.7 


0.25 


Poland 


0.7 


0.16 


3.1 


0.32 


95.7 


0.42 


t 


t 


Portugal 


4.2 


0.64 


10.6 


0.90 


20.3 


1.62 


64.3 


2.39 


Slovak Republic 


0.6 


0.22 


0.9 


0.23 


37.1 


1.56 


60.9 


1.54 


Spain 


# 


t 


3.2 


0.36 


27.0 


0.85 


69.7 


1.01 


Sweden 


# 


t 


2.4 


0.21 


93.0 


0.97 


4.6 


0.93 


Switzerland 


0.7 


0.16 


16.9 


1.01 


62.7 


2.06 


19.3 


2.50 


Turkey 


t 


t 


4.4 


1.60 


3.2 


0.67 


52.1 


2.18 


United States 


t 


t 


2.4 


0.61 


29.7 


1.42 


60.6 


1.35 


Non-OECD countries 


Hong Kong-China 


5.1 


0.37 


10.7 


0.80 


25.7 


0.95 


58.4 


0.98 


Indonesia 


2.4 


0.35 


12.7 


0.80 


48.8 


1.69 


34.5 


1.91 


Latvia 


1.0 


0.16 


16.1 


0.86 


73.1 


2.08 


5.9 


0.56 


Liechtenstein 


t 


t 


20.4 


1.00 


71.3 


0.94 


t 


t 


Macao-China 


12.3 


0.50 


25.9 


0.72 


36.8 


0.74 


24.7 


0.60 


Russian Federation 


t 


t 


2.6 


0.34 


28.7 


1.92 


67.2 


2.18 


Serbia and Montenegro 


# 


t 


# 


t 


97.6 


0.35 


2.4 


0.35 


Thailand 


t 


t 


1.1 


0.39 


44.1 


1.20 


53.3 


1.24 


Tunisia 


15.4 


0.55 


22.0 


0.70 


25.1 


0.86 


34.5 


1.38 


Uruguay 


5.7 


0.62 


9.7 


0.84 


18.2 


1.23 


59.4 


1.74 


United Kingdom 1 


# 


t 


# 


t 


# 


t 


33.8 


1.10 



See notes at end of table. 
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Table B-1. Percentage distribution of 15-year-old students, by grade and 
country: 2003 — Continued 



Country 


1 1 th 




12 th 




Grade not reported 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


6.0 


0.13 


0.1 


0.00 


# 


t 


OECD countries 


Australia 


19.2 


0.67 


t 


t 


# 


t 


Austria 


# 


t 


# 


t 


t 


t 


Belgium 


0.8 


0.09 


# 


t 


t 


t 


Canada 


1.2 


0.12 


# 


t 


# 


t 


Czech Republic 


# 


t 


# 


t 


# 


t 


Denmark 


# 


t 


# 


t 


# 


t 


Finland 


# 


t 


# 


t 


# 


t 


France 


2.2 


0.30 


# 


t 


# 


t 


Germany 


t 


t 


# 


t 


t 


t 


Greece 


15.0 


0.90 


# 


t 


# 


t 


Hungary 


# 


t 


# 


t 


# 


t 


Iceland 


# 


t 


# 


t 


# 


t 


Ireland 


19.6 


1.36 


# 


t 


# 


t 


Italy 


4.3 


0.48 


# 


t 


# 


t 


Japan 


# 


t 


# 


t 


# 


t 


Korea, Republic of 


0.1 


t 


# 


t 


# 


t 


Luxembourg 


t 


t 


# 


t 


# 


t 


Mexico 


0.9 


0.49 


# 


t 


# 


t 


Netherlands 


t 


t 


# 


t 


# 


t 


New Zealand 


3.7 


0.24 


# 


t 


t 


t 


Norway 


t 


t 


# 


t 


# 


t 


Poland 


# 


t 


# 


t 


# 


t 


Portugal 


0.6 


t 


# 


t 


# 


t 


Slovak Republic 


0.5 


0.22 


# 


t 


# 


t 


Spain 


# 


t 


# 


t 


# 


t 


Sweden 


# 


t 


# 


t 


# 


t 


Switzerland 


0.2 


t 


# 


t 


t 


t 


Turkey 


39.2 


2.37 


t 


t 


# 


t 


United States 


7.0 


0.89 


# 


t 


# 


t 


Non-OECD countries 


Hong Kong-China 


t 


t 


# 


t 


# 


t 


Indonesia 


1.6 


0.21 


t 


t 


# 


t 


Latvia 


t 


t 


# 


t 


3.7 


2.49 


Liechtenstein 


# 


t 


# 


t 


# 


t 


Macao-China 


t 


t 


# 


t 


# 


t 


Russian Federation 


1.1 


0.18 


# 


t 


# 


t 


Serbia and Montenegro 


# 


t 


# 


t 


# 


t 


Thailand 


1.4 


0.26 


# 


t 


# 


t 


Tunisia 


2.9 


0.24 


# 


t 


# 


t 


Uruguay 


7.1 


1.03 


# 


t 


# 


t 


United Kingdom 1 


63.6 


1.08 


2.6 


0.08 


# 


t 



f Not applicable. 

# Rounds to zero, 
t Reporting standards not met. 

'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: Detail may not sum to totals because of rounding, s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 
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Table B-2. Percentage distribution and average 

combined mathematics literacy scores 
of U.S. 15-year-old students, by type of 
mathematics class: 2003 



Type of class 


Percent 


s.e. 


Average 


s.e 


Pre-algebra or 
general mathematics 


8.7 


0.80 


419.5 


4.97 


Algebra 1 


28.6 


1.01 


442.1 


3.28 


Geometry 


31.1 


1.18 


498.4 


3.47 


Algebra II 


20.7 


1.01 


537.2 


3.67 


Precalculus or calculus 


3.1 


0.39 


595.6 


7.53 


Other 


7.7 


0.68 


482.8 


8.57 



NOTE:Type of class refers to the mathematics class in which the student 
was enrolled at time of the assessment. Detail may not sum to totals 
because of rounding, s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development 
(OECD), Program for International Student Assessment (PISA), 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-3. Average mathematics literacy scores and subscale scores of 15-year-old students, by country: 2003 





Combined 

mathematics 

literacy 








Mathematics subscales 








Country 




Space and shape 


Change and 
relationships 


Quantity 




Uncertainty 






Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


OECD average 


500.0 


0.63 


496.3 


0.65 


498.8 


0.70 


500.7 


0.63 


502.0 


0.61 


OECD countries 


Australia 


524.3 


2.15 


520.6 


2.33 


525.3 


2.30 


516.9 


2.06 


530.9 


2.21 


Austria 


505.6 


3.27 


515.2 


3.48 


499.8 


3.60 


513.2 


3.00 


493.8 


3.13 


Belgium 


529.3 


2.29 


529.6 


2.26 


535.3 


2.45 


529.6 


2.31 


525.7 


2.21 


Canada 


532.5 


1.82 


517.8 


1.81 


536.7 


1.93 


528.1 


1.85 


541.6 


1.83 


Czech Republic 


516.5 


3.55 


527.4 


4.12 


514.8 


3.50 


528.0 


3.54 


500.3 


3.11 


Denmark 


514.3 


2.74 


512.4 


2.76 


509.3 


2.99 


515.6 


2.64 


515.6 


2.78 


Finland 


544.3 


1.87 


539.0 


2.04 


543.1 


2.19 


548.5 


1.83 


544.8 


2.09 


France 


510.8 


2.50 


507.6 


2.98 


519.7 


2.62 


506.9 


2.49 


506.1 


2.39 


Germany 


503.0 


3.32 


499.6 


3.28 


507.2 


3.73 


513.8 


3.37 


492.5 


3.29 


Greece 


444.9 


3.90 


437.1 


3.80 


435.6 


4.31 


445.9 


3.97 


458.4 


3.53 


Hungary 


490.0 


2.84 


479.1 


3.34 


494.6 


3.10 


496.3 


2.72 


489.0 


2.63 


Iceland 


515.1 


1.42 


503.5 


1.46 


509.5 


1.43 


513.3 


1.50 


527.8 


1.50 


Ireland 


502.8 


2.45 


476.2 


2.43 


506.0 


2.45 


501.7 


2.48 


517.2 


2.65 


Italy 


465.6 


3.08 


470.3 


3.14 


452.1 


3.21 


474.8 


3.38 


462.6 


3.03 


Japan 


534.1 


4.02 


553.2 


4.31 


536.1 


4.33 


526.6 


3.79 


527.9 


3.88 


Korea, Republic of 


542.2 


3.24 


551.7 


3.80 


547.6 


3.52 


537.2 


2.97 


538.3 


3.03 


Luxembourg 


493.2 


0.97 


488.2 


1.35 


487.0 


1.15 


501.5 


1.06 


492.1 


1.06 


Mexico 


385.2 


3.64 


381.7 


3.20 


364.1 


4.14 


393.8 


3.94 


389.8 


3.26 


Netherlands 


537.8 


3.13 


526.1 


2.87 


551.4 


3.12 


528.3 


3.09 


549.3 


2.99 


New Zealand 


523.5 


2.26 


524.9 


2.34 


525.7 


2.37 


511.1 


2.22 


532.2 


2.30 


Norway 


495.2 


2.38 


482.7 


2.54 


487.7 


2.64 


494.2 


2.22 


512.8 


2.59 


Poland 


490.2 


2.50 


490.3 


2.66 


484.3 


2.70 


491.8 


2.47 


493.5 


2.35 


Portugal 


466.0 


3.40 


450.2 


3.43 


467.9 


3.95 


465.4 


3.51 


470.6 


3.41 


Slovak Republic 


498.2 


3.35 


505.4 


4.01 


494.4 


3.48 


512.5 


3.43 


475.8 


3.21 


Spain 


485.1 


2.41 


476.5 


2.59 


480.7 


2.80 


492.3 


2.53 


489.0 


2.42 


Sweden 


509.0 


2.56 


498.3 


2.56 


505.1 


2.94 


513.6 


2.49 


510.8 


2.72 


Switzerland 


526.6 


3.38 


539.5 


3.50 


522.7 


3.65 


532.6 


3.08 


516.5 


3.28 


Turkey 


423.4 


6.74 


417.4 


6.35 


422.8 


7.57 


413.2 


6.78 


442.6 


6.21 


United States 


482.9 


2.95 


472.0 


2.78 


485.5 


3.03 


476.4 


3.18 


491.5 


2.97 


Non-OECD countries 


Hong Kong-China 


550.4 


4.54 


558.4 


4.85 


539.7 


4.68 


545.2 


4.19 


558.3 


4.56 


Indonesia 


360.2 


3.91 


360.8 


3.70 


333.9 


4.58 


357.5 


4.28 


384.5 


2.86 


Latvia 


483.4 


3.69 


486.4 


4.04 


487.2 


4.36 


481.7 


3.60 


473.8 


3.28 


Liechtenstein 


535.8 


4.12 


538.2 


4.58 


539.5 


3.67 


533.5 


4.12 


523.4 


3.68 


Macao-China 


527.3 


2.89 


527.9 


3.29 


518.8 


3.53 


533.0 


2.99 


531.6 


3.21 


Russian Federation 


468.4 


4.20 


474.3 


4.69 


476.8 


4.64 


472.4 


4.04 


436.5 


4.02 


Serbia and Montenegro 


436.9 


3.75 


432.5 


3.91 


419.0 


3.97 


456.3 


3.79 


427.9 


3.49 


Thailand 


417.0 


3.00 


423.9 


3.35 


405.0 


3.39 


414.8 


3.15 


422.7 


2.53 


Tunisia 


358.7 


2.54 


358.9 


2.56 


336.6 


2.78 


364.4 


2.79 


363.3 


2.30 


Uruguay 


422.2 


3.29 


412.0 


2.98 


417.0 


3.60 


429.7 


3.22 


418.6 


3.11 


United Kingdom 1 


508.3 


2.43 


496.0 


2.50 


512.9 


2.54 


498.5 


2.52 


520.1 


2.41 



'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the national averages of the OECD member countries with data available. Because PISA is principally an 
OECD study, the results for non-OECD countries are displayed separately from those of the OECD countries and are not included in the OECD average, 
s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2003. 
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Table B-4. Combined mathematics literacy scores of 15-year-old students, by percentiles 
and country: 2003 



Country 


5 th percentile 


10 th percentile 


25 th percentile 


50 th percentile 


Score 


s.e. 


Score 


s.e. 


Score 


s.e. 


Score 


s.e. 


OECD average 


331.7 


1.30 


369.0 


i.ii 


432.4 


0.93 


501.1 


0.26 


OECD countries 


Australia 


364.3 


4.44 


398.6 


3.43 


459.8 


2.75 


527.5 


0.68 


Austria 


353.4 


6.64 


384.4 


4.44 


439.4 


4.02 


506.2 


0.91 


Belgium 


333.8 


6.53 


380.7 


4.61 


456.2 


3.44 


537.1 


1.48 


Canada 


386.2 


3.05 


419.3 


2.54 


473.9 


2.19 


534.0 


0.62 


Czech Republic 


358.0 


6.25 


391.7 


5.72 


449.4 


4.55 


516.8 


1.46 


Denmark 


360.7 


4.39 


395.8 


4.53 


453.2 


3.67 


516.1 


1.54 


Finland 


406.4 


3.83 


438.0 


2.77 


488.2 


2.21 


543.7 


0.88 


France 


352.3 


5.96 


388.7 


5.56 


449.1 


3.74 


514.2 


0.86 


Germany 


324.0 


6.08 


363.0 


5.65 


432.2 


4.66 


509.2 


0.97 


Greece 


287.6 


5.39 


323.5 


5.13 


382.4 


4.57 


446.1 


21.6 


Hungary 


335.3 


5.62 


369.6 


4.23 


426.1 


3.01 


490.0 


0.94 


Iceland 


362.4 


4.05 


396.1 


2.74 


454.2 


2.81 


518.2 


1.30 


Ireland 


360.4 


4.68 


393.1 


3.21 


445.0 


3.38 


503.5 


0.72 


Italy 


307.2 


6.39 


342.4 


5.86 


400.5 


4.34 


466.1 


1.11 


Japan 


360.9 


8.24 


401.7 


6.26 


467.2 


5.37 


539.5 


1.66 


Korea, Republic of 


387.8 


4.61 


422.8 


4.46 


479.3 


3.74 


544.1 


1.69 


Luxembourg 


338.5 


3.87 


372.7 


2.69 


430.2 


2.15 


495.2 


0.52 


Mexico 


247.1 


5.39 


276.1 


4.70 


326.6 


4.32 


384.7 


1.14 


Netherlands 


385.2 


6.86 


415.4 


5.84 


470.9 


5.44 


539.6 


1.51 


New Zealand 


358.5 


4.07 


394.3 


3.89 


455.2 


2.91 


525.3 


1.38 


Norway 


343.5 


3.96 


375.9 


3.42 


432.9 


2.87 


495.3 


1.30 


Poland 


343.4 


5.78 


376.0 


3.62 


428.2 


3.13 


489.7 


1.18 


Portugal 


320.9 


6.26 


351.9 


5.25 


406.0 


4.96 


467.2 


2.40 


Slovak Republic 


342.4 


6.91 


378.5 


5.81 


435.6 


4.57 


498.3 


1.28 


Spain 


335.0 


5.13 


368.6 


3.54 


426.2 


2.98 


487.1 


1.12 


Sweden 


352.7 


5.29 


387.1 


4.38 


446.1 


3.02 


509.6 


1.69 


Switzerland 


358.7 


4.80 


395.7 


4.16 


460.8 


3.57 


529.7 


0.97 


Turkey 


269.7 


5.76 


300.2 


5.01 


350.8 


5.26 


414.5 


1.75 


United States 


323.0 


4.88 


356.5 


4.55 


418.0 


3.69 


483.5 


1.36 


Non-OECD countries 


Hong Kong-China 


373.8 


11.05 


417.0 


8.02 


484.8 


6.91 


559.5 


1.49 


Indonesia 


233.2 


5.22 


260.5 


4.81 


306.0 


3.49 


356.9 


2.06 


Latvia 


339.2 


5.90 


370.8 


5.14 


423.5 


3.90 


484.1 


1.19 


Liechtenstein 


361.9 


19.68 


408.0 


9.77 


469.9 


7.58 


539.2 


6.71 


Macao-China 


382.3 


8.76 


414.5 


5.97 


467.2 


4.41 


529.0 


1.66 


Russian Federation 


318.5 


5.46 


350.8 


4.96 


405.8 


4.83 


467.1 


1.29 


Serbia and Montenegro 


298.9 


4.37 


328.6 


4.47 


378.6 


3.96 


436.6 


2.56 


Thailand 


289.9 


3.95 


316.3 


3.10 


360.5 


2.92 


411.9 


1.96 


Tunisia 


228.6 


3.80 


256.3 


3.51 


303.0 


2.55 


355.5 


0.77 


Uruguay 


255.3 


4.30 


291.3 


3.80 


353.3 


4.07 


425.0 


1.71 


United Kingdom 1 


356.1 


4.92 


387.8 


4.05 


444.1 


3.18 


509.8 


0.99 



See notes at end of table. 
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Table B-4. Combined mathematics literacy scores of 15-year-old students, 
by percentiles and country: 2003 — Continued 



Country 


75 th percentile 


90" 1 percentile 


95 th percentile 


Score 


s.e. 


Score 


s.e. 


Score 


s.e. 


OECD average 


570.5 


0.70 


628.3 


0.74 


660.2 


0.95 


OECD countries 


Australia 


591.6 


2.50 


644.7 


3.04 


675.7 


3.53 


Austria 


571.4 


4.18 


626.2 


3.96 


658.2 


4.96 


Belgium 


611.2 


2.49 


664.4 


2.35 


693.4 


2.38 


Canada 


593.3 


2.13 


644.2 


2.58 


672.7 


3.39 


Czech Republic 


584.4 


3.98 


641.0 


4.35 


671.9 


4.89 


Denmark 


578.2 


3.14 


631.5 


3.65 


662.0 


4.73 


Finland 


602.6 


2.32 


651.7 


2.83 


680.2 


3.13 


France 


575.2 


3.01 


627.7 


3.58 


656.2 


3.46 


Germany 


578.3 


3.48 


632.3 


3.50 


661.7 


3.64 


Greece 


507.9 


4.28 


565.9 


5.25 


597.8 


5.10 


Hungary 


555.9 


3.90 


610.7 


4.71 


643.8 


4.59 


Iceland 


578.4 


1.95 


629.2 


3.02 


657.9 


3.77 


Ireland 


561.9 


3.01 


613.9 


3.59 


641.0 


3.30 


Italy 


530.2 


3.01 


589.1 


3.63 


623.2 


3.74 


Japan 


605.1 


4.36 


659.6 


6.14 


690.2 


6.58 


Korea, Republic of 


606.1 


4.22 


659.2 


5.37 


690.2 


6.83 


Luxembourg 


557.2 


1.91 


611.4 


3.20 


641.4 


2.72 


Mexico 


443.6 


4.46 


497.1 


4.69 


526.9 


5.65 


Netherlands 


608.3 


3.84 


656.5 


3.21 


683.5 


3.43 


New Zealand 


593.0 


2.21 


650.0 


3.20 


682.3 


2.91 


Norway 


560.0 


3.32 


613.6 


3.56 


644.7 


3.92 


Poland 


552.8 


2.87 


607.4 


3.34 


639.9 


3.50 


Portugal 


526.1 


3.52 


579.9 


3.29 


609.9 


3.72 


Slovak Republic 


564.6 


3.78 


619.1 


3.49 


648.4 


4.07 


Spain 


546.4 


3.12 


597.4 


3.50 


626.0 


3.70 


Sweden 


575.6 


3.19 


630.5 


3.82 


661.9 


4.80 


Switzerland 


595.0 


4.89 


652.1 


5.23 


684.0 


6.84 


Turkey 


484.9 


8.53 


559.7 


14.23 


613.6 


22.75 


United States 


549.7 


3.36 


607.4 


3.87 


638.0 


5.14 


Non-OECD countries 


Hong Kong-China 


621.8 


3.74 


671.8 


4.10 


699.5 


3.97 


Indonesia 


411.5 


4.77 


465.8 


6.50 


498.8 


7.69 


Latvia 


543.5 


4.72 


596.4 


4.43 


626.3 


4.97 


Liechtenstein 


608.6 


7.91 


655.3 


9.53 


686.4 


16.38 


Macao-China 


587.3 


4.01 


639.1 


5.48 


668.4 


8.28 


Russian Federation 


530.1 


4.95 


588.1 


5.28 


622.4 


6.10 


Serbia and Montenegro 


493.1 


4.78 


546.4 


5.05 


579.2 


5.29 


Thailand 


469.3 


3.75 


526.0 


4.70 


560.0 


6.43 


Tunisia 


411.6 


3.59 


465.8 


4.78 


501.4 


6.80 


Uruguay 


490.7 


3.77 


550.0 


4.36 


583.4 


4.67 


United Kingdom 1 


572.6 


3.18 


628.7 


3.55 


659.3 


4.79 



'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the national averages of the OECD member countries with 
data available. Because PISA is principally an OECD study, the results for non-OECD countries are dis- 
played separately from those of the OECD countries and are not included in the OECD average, s.e. 
means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 
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Table B-5. Standard deviations of 

15-year-old students' combined 
mathematics literacy scores, 
by country: 2003 



Country 


Standard 

deviation 


s.e. 


OECD average 


100.0 


0.41 


OECD countries 


Australia 


95.4 


1.50 


Austria 


93.1 


1.67 


Belgium 


109.9 


1.78 


Canada 


87.1 


0.97 


Czech Republic 


95.9 


1.87 


Denmark 


91.3 


1.44 


Finland 


83.7 


1.08 


France 


91.7 


1.80 


Germany 


102.6 


1.77 


Greece 


93.8 


1.76 


Hungary 


93.5 


1.96 


Iceland 


90.4 


1.21 


Ireland 


85.3 


1.26 


Italy 


95.7 


1.87 


Japan 


100.5 


2.75 


Korea, Republic of 


92.4 


2.14 


Luxembourg 


91.9 


0.95 


Mexico 


85.4 


1.85 


Netherlands 


92.5 


2.33 


New Zealand 


98.3 


1.17 


Norway 


92.0 


1.15 


Poland 


90.2 


1.34 


Portugal 


87.6 


1.66 


Slovak Republic 


93.3 


2.32 


Spain 


88.5 


1.26 


Sweden 


94.7 


1.79 


Switzerland 


98.4 


2.05 


Turkey 


104.7 


5.34 


United States 


95.2 


1.29 


Non-OECD countries 


Hong Kong-China 


100.2 


3.01 


Indonesia 


80.5 


2.06 


Latvia 


87.9 


1.66 


Liechtenstein 


99.1 


4.43 


Macao-China 


86.9 


2.41 


Russian Federation 


92.3 


1.93 


Serbia and Montenegro 


84.7 


1.55 


Thailand 


82.0 


1.79 


Tunisia 


82.0 


1.95 


Uruguay 


99.7 


1.60 


United Kingdom 1 


92.3 


1.35 



'Due to low response rates, data for the United Kingdom are 
not discussed in this report. 

NOTE:The OECD average is the average of the national 
averages of the OECD member countries with data available. 
Because PISA is principally an OECD study, the results for 
non-OECD countries are displayed separately from those of 
the OECD countries and are not included in the OECD aver- 
age. s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and 
Development (OECD), Program for International Student 
Assessment (PISA), 2003. 
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Table B-6. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the combined mathematics literacy scale, by country: 2003 



Country 


Below level 1 




Level 1 




Level 2 




Level 3 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


8.2 


0.17 


13.2 


0.16 


21.1 


0.15 


23.7 


0.18 


OECD countries 


Australia 


4.3 


0.45 


10.0 


0.51 


18.6 


0.62 


24.0 


0.71 


Austria 


5.6 


0.70 


13.2 


0.84 


21.6 


0.90 


24.9 


1.14 


Belgium 


7.2 


0.56 


9.3 


0.49 


15.9 


0.65 


20.1 


0.71 


Canada 


2.4 


0.26 


7.7 


0.36 


18.3 


0.61 


26.2 


0.67 


Czech Republic 


5.0 


0.69 


11.6 


0.90 


20.1 


0.96 


24.3 


0.95 


Denmark 


4.7 


0.50 


10.7 


0.62 


20.6 


0.89 


26.2 


0.88 


Finland 


1.5 


0.23 


5.3 


0.38 


16.0 


0.57 


27.7 


0.65 


France 


5.6 


0.68 


11.0 


0.77 


20.2 


0.82 


25.9 


0.99 


Germany 


9.2 


0.84 


12.4 


0.81 


19.0 


1.05 


22.6 


0.82 


Greece 


17.8 


1.21 


21.2 


1.15 


26.3 


1.04 


20.2 


1.01 


Hungary 


7.8 


0.80 


15.2 


0.81 


23.8 


1.05 


24.3 


0.93 


Iceland 


4.5 


0.40 


10.5 


0.55 


20.2 


1.02 


26.1 


0.88 


Ireland 


4.7 


0.57 


12.1 


0.84 


23.6 


0.83 


28.0 


0.82 


Italy 


13.2 


1.19 


18.7 


0.93 


24.7 


1.03 


22.9 


0.84 


Japan 


4.7 


0.65 


8.6 


0.72 


16.3 


0.80 


22.4 


1.02 


Korea, Republic of 


2.5 


0.32 


7.1 


0.65 


16.6 


0.80 


24.1 


0.98 


Luxembourg 


7.4 


0.41 


14.3 


0.65 


22.9 


0.87 


25.9 


0.79 


Mexico 


38.1 


1.71 


27.9 


1.02 


20.8 


0.87 


10.1 


0.84 


Netherlands 


2.6 


0.65 


8.4 


0.95 


18.0 


1.11 


23.0 


1.14 


New Zealand 


4.9 


0.44 


10.1 


0.63 


19.2 


0.71 


23.2 


0.90 


Norway 


6.9 


0.50 


13.9 


0.82 


23.7 


1.16 


25.2 


1.01 


Poland 


6.8 


0.61 


15.2 


0.76 


24.8 


0.75 


25.3 


0.94 


Portugal 


11.3 


1.11 


18.8 


0.99 


27.1 


0.99 


24.0 


1.03 


Slovak Republic 


6.7 


0.85 


13.2 


0.86 


23.5 


0.88 


24.9 


1.08 


Spain 


8.1 


0.66 


14.9 


0.87 


24.7 


0.78 


26.7 


1.02 


Sweden 


5.6 


0.52 


11.7 


0.60 


21.7 


0.84 


25.5 


0.95 


Switzerland 


4.9 


0.45 


9.6 


0.57 


17.5 


0.80 


24.3 


0.98 


Turkey 


27.7 


2.01 


24.6 


1.33 


22.1 


1.12 


13.5 


1.27 


United States 


10.2 


0.80 


15.5 


0.81 


23.9 


0.80 


23.8 


0.79 


Non-OECD countries 


Hong Kong-China 


3.9 


0.72 


6.5 


0.64 


13.9 


1.00 


20.0 


1.25 


Indonesia 


50.5 


2.08 


27.6 


1.05 


14.8 


1.07 


5.5 


0.71 


Latvia 


7.6 


0.86 


16.1 


1.08 


25.5 


1.17 


26.3 


1.15 


Liechtenstein 


4.8 


1.33 


7.5 


1.66 


17.3 


2.78 


21.6 


2.54 


Macao-China 


2.3 


0.60 


8.8 


1.34 


19.6 


1.40 


26.8 


1.77 


Russian Federation 


11.4 


1.03 


18.8 


1.09 


26.4 


1.13 


23.1 


1.02 


Serbia and Montenegro 


17.6 


1.35 


24.5 


1.08 


28.6 


1.16 


18.9 


1.11 


Thailand 


23.8 


1.28 


30.2 


1.25 


25.4 


1.12 


13.7 


0.85 


Tunisia 


51.1 


1.37 


26.9 


0.95 


14.7 


0.75 


5.7 


0.61 


Uruguay 


26.3 


1.30 


21.8 


0.80 


24.2 


0.89 


16.8 


0.68 


United Kingdom 1 


5.2 


0.54 


12.5 


0.67 


21.2 


1.20 


25.6 


0.88 



See notes at end of table. 



PISA 2003 Results From the U.S. Perspective 



Table B-6. Percentage distribution of 15-year-old students scoring at each 
proficiency level on the combined mathematics literacy scale, by 
country: 2003 — Continued 



Country 


Level 4 




Level 5 




Level 6 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


19.1 


0.17 


10.6 


0.13 


4.0 


0.10 


OECD countries 


Australia 


23.3 


0.64 


14.0 


0.53 


5.8 


0.45 


Austria 


20.5 


0.84 


10.5 


0.85 


3.7 


0.52 


Belgium 


21.0 


0.62 


17.5 


0.69 


9.0 


0.48 


Canada 


25.1 


0.60 


14.8 


0.55 


5.5 


0.45 


Czech Republic 


20.8 


0.87 


12.9 


0.80 


5.3 


0.53 


Denmark 


21.9 


0.83 


11.8 


0.86 


4.1 


0.50 


Finland 


26.1 


0.89 


16.7 


0.64 


6.7 


0.46 


France 


22.1 


0.97 


11.6 


0.72 


3.5 


0.40 


Germany 


20.6 


1.02 


12.2 


0.87 


4.1 


0.48 


Greece 


10.6 


0.87 


3.4 


0.53 


0.6 


0.17 


Hungary 


18.2 


0.90 


8.2 


0.73 


2.5 


0.42 


Iceland 


23.2 


0.81 


11.7 


0.61 


3.7 


0.36 


Ireland 


20.2 


1.06 


9.1 


0.76 


2.2 


0.33 


Italy 


13.4 


0.73 


5.5 


0.43 


1.5 


0.19 


Japan 


23.6 


1.24 


16.1 


0.96 


8.2 


1.14 


Korea, Republic of 


25.0 


1.08 


16.7 


0.81 


8.1 


0.93 


Luxembourg 


18.7 


0.85 


8.5 


0.59 


2.4 


0.31 


Mexico 


2.7 


0.39 


0.4 


0.10 


# 


t 


Netherlands 


22.6 


1.34 


18.2 


1.09 


7.3 


0.58 


New Zealand 


21.9 


0.80 


14.1 


0.60 


6.6 


0.44 


Norway 


18.9 


1.00 


8.7 


0.57 


2.7 


0.35 


Poland 


17.7 


0.89 


7.8 


0.49 


2.3 


0.31 


Portugal 


13.4 


0.94 


4.6 


0.47 


0.8 


0.16 


Slovak Republic 


18.9 


0.82 


9.8 


0.68 


2.9 


0.38 


Spain 


17.7 


0.65 


6.5 


0.62 


1.4 


0.25 


Sweden 


19.8 


0.81 


11.6 


0.57 


4.1 


0.49 


Switzerland 


22.5 


0.72 


14.2 


1.05 


7.0 


0.90 


Turkey 


6.8 


1.05 


3.1 


0.82 


2.4 


1.02 


United States 


16.6 


0.73 


8.0 


0.53 


2.0 


0.36 


Non-OECD countries 


Hong Kong-China 


25.0 


1.17 


20.2 


1.00 


10.5 


0.94 


Indonesia 


1.4 


0.39 


0.2 


0.09 


# 


t 


Latvia 


16.6 


1.17 


6.3 


0.70 


1.6 


0.36 


Liechtenstein 


23.2 


3.09 


18.3 


3.22 


7.3 


1.73 


Macao-China 


23.7 


1.71 


13.8 


1.55 


4.8 


0.96 


Russian Federation 


13.2 


0.92 


5.4 


0.58 


1.6 


0.38 


Serbia and Montenegro 


8.1 


0.88 


2.1 


0.41 


0.2 


0.10 


Thailand 


5.3 


0.53 


1.5 


0.31 


0.2 


0.10 


Tunisia 


1.4 


0.30 


0.2 


0.12 


# 


t 


Uruguay 


8.2 


0.65 


2.3 


0.33 


0.5 


0.17 


United Kingdom 1 


20.6 


0.73 


11.0 


0.73 


3.9 


0.43 



f Not applicable. 

# Rounds to zero. 

'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a 
majority of items at that level. Students were classified into mathematics literacy levels according to their 
scores. Exact cut point scores are as follows: below level 1 (a score less than or equal to 357.77)\ level 1 (a 
score greater than 357.77 and less than or equal to 420.07 ); level 2 (a score greater than 420.07 and less than or 
equal to 482.38 ); level 3 (a score greater than 482.38 and less than or equal to 544.68)\ level 4 (a score greater 
than 544.68 and less than or equal to 606.99 ); level 5 (a score greater than 606.99 and less than or equal to 
669. 3); level 6 (a score greater than 669.3). the OECD average is the average of the national averages of the 
OECD member countries with data available. Because PISA is principally an OECD study, the results for 
non-OECD countries are displayed separately from those of the OECD countries and are not included in 
the OECD average, s.e. means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-7. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the mathematics literacy quantity subscale, by country: 2003 



Country 


Below level 1 




Level 1 




Level 2 




Level 3 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


8.8 


0.18 


12.5 


0.15 


20.1 


0.15 


23.7 


0.21 


OECD countries 


Australia 


5.5 


0.44 


11.0 


0.47 


19.0 


0.76 


24.3 


0.90 


Austria 


3.7 


0.49 


11.2 


0.88 


20.9 


1.00 


27.2 


1.06 


Belgium 


7.2 


0.57 


8.9 


0.53 


15.1 


0.52 


20.6 


0.62 


Canada 


3.8 


0.29 


8.8 


0.37 


18.1 


0.64 


25.2 


0.64 


Czech Republic 


4.7 


0.72 


9.7 


0.86 


17.2 


0.89 


23.5 


0.95 


Denmark 


4.7 


0.59 


10.4 


0.58 


19.9 


0.79 


26.3 


0.90 


Finland 


1.4 


0.21 


5.0 


0.53 


14.6 


0.66 


26.9 


0.69 


France 


6.7 


0.74 


11.1 


0.82 


20.4 


0.98 


25.4 


1.15 


Germany 


8.5 


0.70 


10.4 


0.83 


17.5 


0.90 


22.0 


1.08 


Greece 


19.0 


1.21 


19.8 


0.88 


25.1 


0.89 


20.0 


0.92 


Hungary 


7.8 


0.73 


13.5 


0.84 


21.6 


0.90 


25.2 


0.89 


Iceland 


6.2 


0.43 


10.9 


0.63 


19.1 


1.09 


24.3 


1.00 


Ireland 


5.6 


0.57 


12.3 


0.85 


23.0 


1.00 


26.9 


1.06 


Italy 


13.7 


1.06 


16.1 


0.74 


22.0 


0.76 


22.4 


0.82 


Japan 


5.7 


0.73 


9.2 


0.78 


16.6 


0.81 


23.1 


1.12 


Korea, Republic of 


2.6 


0.31 


7.2 


0.70 


17.0 


0.84 


25.2 


0.84 


Luxembourg 


6.5 


0.44 


12.4 


0.78 


21.8 


0.99 


26.2 


1.35 


Mexico 


35.5 


1.80 


25.0 


1.23 


21.4 


1.12 


12.4 


0.77 


Netherlands 


4.1 


0.71 


10.1 


0.96 


18.3 


1.22 


23.0 


1.22 


New Zealand 


6.4 


0.55 


11.9 


0.70 


20.1 


0.71 


23.6 


0.78 


Norway 


7.7 


0.54 


13.8 


0.74 


22.8 


0.93 


25.4 


1.06 


Poland 


7.1 


0.68 


13.5 


0.70 


24.2 


0.96 


27.1 


0.87 


Portugal 


12.9 


1.22 


18.3 


1.06 


25.2 


0.80 


23.4 


1.22 


Slovak Republic 


5.6 


0.75 


10.6 


0.77 


20.0 


0.80 


26.1 


0.89 


Spain 


8.9 


0.66 


13.2 


0.88 


22.5 


0.78 


25.0 


0.66 


Sweden 


4.4 


0.45 


10.3 


0.60 


21.4 


0.81 


27.3 


1.00 


Switzerland 


4.2 


0.38 


8.6 


0.60 


16.0 


0.81 


24.2 


1.03 


Turkey 


32.1 


2.07 


23.1 


1.03 


20.2 


1.07 


12.6 


1.06 


United States 


13.7 


1.00 


15.6 


0.76 


22.0 


0.70 


21.9 


0.78 


Non-OECD countries 


Hong Kong-China 


4.1 


0.70 


7.0 


0.67 


13.7 


1.24 


21.5 


1.33 


Indonesia 


51.5 


1.92 


24.7 


0.93 


14.9 


1.03 


6.1 


0.64 


Latvia 


7.4 


0.87 


15.5 


1.20 


26.4 


1.13 


27.7 


1.21 


Liechtenstein 


4.0 


1.37 


7.6 


1.36 


16.5 


2.88 


24.1 


2.90 


Macao-China 


2.4 


0.62 


8.1 


1.29 


17.8 


1.37 


25.8 


1.71 


Russian Federation 


11.1 


1.05 


16.8 


1.01 


25.8 


0.87 


24.6 


1.03 


Serbia and Montenegro 


13.6 


1.11 


20.6 


1.15 


27.1 


1.15 


22.1 


1.13 


Thailand 


27.7 


1.39 


26.4 


1.21 


23.3 


0.91 


13.7 


0.84 


Tunisia 


49.0 


1.31 


25.2 


0.99 


16.1 


0.91 


7.0 


0.59 


Uruguay 


25.6 


1.11 


19.5 


0.82 


22.1 


0.81 


18.1 


1.17 


United Kingdom 1 


8.3 


0.64 


13.7 


0.67 


20.7 


0.96 


24.2 


0.68 



See notes at end of table. 



PISA 2003 Results From the U.S. Perspective 



Table B-7. Percentage distribution of 15-year-old students scoring at each 
proficiency level on the mathematics literacy quantity subscale, 
by country: 2003 — Continued 



Country 


Level 4 




Level 5 




Level 6 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


19.9 


0.17 


11.0 


0.11 


4.0 


0.09 


OECD countries 


Australia 


22.4 


0.64 


12.5 


0.57 


5.2 


0.37 


Austria 


23.1 


0.99 


11.2 


0.81 


2.8 


0.45 


Belgium 


22.3 


0.59 


17.5 


0.61 


8.5 


0.45 


Canada 


23.7 


0.52 


14.4 


0.52 


6.0 


0.35 


Czech Republic 


23.1 


0.89 


15.0 


0.74 


6.7 


0.62 


Denmark 


22.7 


0.89 


12.0 


0.71 


4.0 


0.42 


Finland 


27.3 


0.89 


17.9 


0.65 


7.0 


0.44 


France 


21.9 


0.84 


11.0 


0.71 


3.5 


0.34 


Germany 


22.0 


1.20 


14.1 


0.98 


5.5 


0.43 


Greece 


11.0 


0.80 


4.1 


0.56 


1.0 


0.29 


Hungary 


19.7 


0.80 


9.7 


0.67 


2.5 


0.31 


Iceland 


22.5 


0.83 


12.7 


0.68 


4.2 


0.52 


Ireland 


20.6 


0.84 


9.5 


0.62 


2.2 


0.36 


Italy 


15.2 


0.82 


7.7 


0.49 


2.8 


0.27 


Japan 


23.6 


1.03 


15.1 


0.84 


6.7 


0.85 


Korea, Republic of 


26.0 


0.98 


15.6 


0.86 


6.4 


0.79 


Luxembourg 


21.0 


0.82 


9.4 


0.59 


2.7 


0.29 


Mexico 


4.6 


0.48 


1.0 


0.21 


0.1 


0.06 


Netherlands 


21.9 


1.11 


15.9 


1.03 


6.7 


0.58 


New Zealand 


21.2 


0.76 


11.9 


0.61 


5.0 


0.33 


Norway 


18.8 


0.91 


8.9 


0.58 


2.6 


0.31 


Poland 


18.7 


0.82 


7.6 


0.62 


1.8 


0.28 


Portugal 


13.8 


0.78 


5.2 


0.42 


1.2 


0.22 


Slovak Republic 


21.9 


0.83 


12.3 


0.76 


3.6 


0.44 


Spain 


18.8 


0.77 


8.8 


0.60 


2.6 


0.25 


Sweden 


21.6 


0.87 


11.1 


0.77 


3.9 


0.56 


Switzerland 


24.6 


0.80 


15.7 


0.95 


6.7 


0.88 


Turkey 


6.5 


1.01 


3.2 


0.71 


2.3 


0.92 


United States 


16.0 


0.67 


8.1 


0.67 


2.8 


0.36 


Non-OECD countries 


Hong Kong-China 


25.8 


1.21 


18.7 


0.95 


9.2 


0.71 


Indonesia 


2.1 


0.49 


0.6 


0.19 


0.1 


0.05 


Latvia 


16.3 


1.12 


5.5 


0.59 


1.2 


0.27 


Liechtenstein 


24.8 


2.63 


17.1 


2.44 


6.0 


1.49 


Macao-China 


25.3 


1.76 


15.6 


1.50 


5.1 


1.06 


Russian Federation 


14.8 


0.99 


5.6 


0.64 


1.4 


0.27 


Serbia and Montenegro 


12.3 


0.99 


3.7 


0.60 


0.7 


0.18 


Thailand 


6.3 


0.56 


2.0 


0.38 


0.6 


0.20 


Tunisia 


2.2 


0.36 


0.4 


0.17 


0.1 


0.05 


Uruguay 


10.0 


0.71 


3.7 


0.40 


0.9 


0.16 


United Kingdom 1 


19.2 


0.73 


10.1 


0.80 


3.8 


0.47 



'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a 
majority of items at that level. Students were classified into mathematics literacy levels according to their 
scores. Exact cut point scores are as follows: below level 1 (a score less than or equal to 357.77)', level 1 (a 
score greater than 357.77 and less than or equal to 420.07)\ level 2 (a score greater than 420.07 and less than or 
equal to 482.38)\ level 3 (a score greater than 482.38 and less than or equal to 544.68)\ level 4 (a score greater 
than 544.68 and less than or equal to 606.99)\ level 5 (a score greater than 606.99 and less than or equal to 
669. 3); level 6 (a score greater than 669.3). The OECD average is the average of the national averages of the 
OECD member countries with data available. Because PISA is principally an OECD study, the results for 
non-OECD countries are displayed separately from those of the OECD countries and are not included in 
the OECD average, s.e. means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-8. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the mathematics literacy space and shape subscale, by country: 2003 



Country 


Below level 1 




Level 1 




Level 2 




Level 3 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


10.6 


0.19 


14.2 


0.16 


20.4 


0.14 


21.5 


0.16 


OECD countries 


Australia 


6.1 


0.50 


10.8 


0.59 


18.4 


0.52 


23.0 


0.66 


Austria 


8.0 


0.74 


12.0 


0.79 


18.6 


0.82 


21.4 


0.74 


Belgium 


6.6 


0.53 


10.4 


0.48 


16.7 


0.51 


20.3 


0.70 


Canada 


4.7 


0.36 


10.7 


0.57 


20.4 


0.61 


25.0 


0.51 


Czech Republic 


8.1 


0.88 


10.6 


0.71 


17.0 


0.72 


19.3 


0.71 


Denmark 


7.1 


0.65 


11.2 


0.74 


19.5 


0.74 


23.8 


0.80 


Finland 


2.5 


0.31 


7.3 


0.47 


17.0 


0.67 


25.5 


0.83 


France 


7.7 


0.77 


12.0 


0.66 


19.6 


0.85 


23.4 


1.07 


Germany 


11.1 


0.77 


13.3 


0.95 


18.6 


0.86 


21.2 


0.94 


Greece 


21.3 


1.17 


21.7 


1.05 


24.4 


1.02 


18.7 


0.95 


Hungary 


13.1 


0.99 


17.3 


0.80 


21.8 


0.76 


20.5 


0.72 


Iceland 


6.5 


0.61 


12.1 


0.66 


21.6 


0.76 


26.0 


1.09 


Ireland 


10.7 


0.78 


16.9 


1.15 


25.4 


0.87 


23.0 


1.02 


Italy 


15.1 


0.99 


16.8 


0.85 


22.0 


0.66 


21.1 


0.70 


Japan 


4.2 


0.65 


7.4 


0.75 


13.9 


0.70 


20.0 


0.84 


Korea, Republic of 


4.8 


0.54 


8.4 


0.57 


14.7 


0.90 


19.7 


0.92 


Luxembourg 


9.5 


0.48 


15.6 


0.62 


23.0 


0.85 


22.6 


1.12 


Mexico 


39.1 


1.59 


27.8 


0.77 


20.6 


0.95 


9.4 


0.71 


Netherlands 


3.7 


0.65 


10.1 


0.78 


18.6 


1.07 


24.9 


1.16 


New Zealand 


5.8 


0.50 


10.8 


0.66 


18.1 


0.84 


21.8 


0.77 


Norway 


11.5 


0.65 


16.1 


0.63 


22.2 


0.92 


22.3 


0.79 


Poland 


10.7 


0.81 


14.9 


0.70 


22.0 


0.93 


22.1 


0.88 


Portugal 


16.4 


1.37 


21.5 


0.78 


26.0 


0.96 


20.2 


0.97 


Slovak Republic 


10.2 


0.92 


13.4 


0.80 


19.0 


0.83 


20.2 


0.80 


Spain 


10.1 


0.76 


16.7 


0.84 


25.5 


0.77 


24.7 


0.83 


Sweden 


7.9 


0.63 


13.4 


0.62 


22.1 


0.84 


24.2 


1.02 


Switzerland 


5.4 


0.52 


8.6 


0.50 


15.7 


0.85 


21.4 


0.94 


Turkey 


28.6 


1.87 


26.0 


1.15 


22.3 


1.18 


12.7 


1.11 


United States 


12.1 


0.84 


18.2 


1.05 


24.7 


1.09 


22.0 


0.95 


Non-OECD countries 


Hong Kong-China 


4.1 


0.65 


7.0 


0.91 


13.2 


1.20 


18.7 


0.94 


Indonesia 


49.7 


1.74 


25.9 


1.22 


15.5 


1.00 


6.6 


0.67 


Latvia 


10.7 


0.93 


15.1 


0.99 


22.4 


0.92 


23.3 


1.14 


Liechtenstein 


5.7 


1.37 


8.1 


1.69 


14.9 


2.79 


21.5 


3.52 


Macao-China 


4.0 


0.71 


9.8 


1.48 


17.6 


2.03 


24.5 


2.01 


Russian Federation 


14.9 


1.03 


16.5 


0.79 


21.9 


0.88 


20.4 


0.84 


Serbia and Montenegro 


21.8 


1.33 


24.4 


1.04 


24.5 


0.77 


16.9 


0.97 


Thailand 


23.4 


1.20 


26.8 


0.95 


24.7 


1.13 


15.4 


0.91 


Tunisia 


49.7 


1.28 


26.0 


1.09 


15.5 


0.66 


6.3 


0.51 


Uruguay 


29.3 


1.15 


23.3 


0.86 


22.9 


0.94 


15.2 


0.79 


United Kingdom 1 


8.6 


0.64 


14.1 


0.98 


21.4 


0.78 


24.3 


0.82 



See notes at end of table. 



PISA 2003 Results From the U.S. Perspective 



Table B-8. Percentage distribution of 15-year-old students scoring at each 
proficiency level on the mathematics literacy space and shape 
subscale, by country: 2003 — Continued 



Country 


Level 4 




Level 5 




Level 6 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


17.2 


0.15 


10.4 


0.11 


5.8 


0.10 


OECD countries 


Australia 


21.2 


0.75 


13.2 


0.56 


7.3 


0.55 


Austria 


19.1 


0.88 


12.3 


0.87 


8.5 


0.74 


Belgium 


20.0 


0.90 


15.7 


0.76 


10.2 


0.47 


Canada 


21.4 


0.49 


12.1 


0.46 


5.6 


0.40 


Czech Republic 


18.9 


0.76 


14.4 


0.80 


11.7 


0.79 


Denmark 


20.0 


0.74 


12.5 


0.69 


5.9 


0.49 


Finland 


24.6 


0.80 


15.2 


0.59 


7.9 


0.60 


France 


20.0 


0.84 


12.0 


0.79 


5.1 


0.51 


Germany 


18.4 


0.82 


11.4 


0.71 


6.0 


0.44 


Greece 


9.6 


0.75 


3.6 


0.50 


0.8 


0.26 


Hungary 


14.8 


0.89 


8.0 


0.74 


4.5 


0.56 


Iceland 


20.5 


0.76 


10.0 


0.63 


3.3 


0.37 


Ireland 


15.4 


0.77 


6.8 


0.64 


1.8 


0.24 


Italy 


14.5 


0.61 


7.2 


0.46 


3.3 


0.28 


Japan 


21.9 


1.00 


18.2 


0.94 


14.3 


1.20 


Korea, Republic of 


19.9 


1.00 


16.5 


0.82 


16.0 


1.27 


Luxembourg 


17.1 


0.73 


8.5 


0.76 


3.6 


0.37 


Mexico 


2.5 


0.37 


0.5 


0.13 


# 


t 


Netherlands 


21.9 


1.10 


14.6 


0.84 


6.2 


0.56 


New Zealand 


20.7 


0.90 


14.4 


0.74 


8.5 


0.49 


Norway 


16.4 


0.73 


8.2 


0.51 


3.3 


0.33 


Poland 


16.4 


0.69 


8.8 


0.53 


5.0 


0.52 


Portugal 


10.9 


0.75 


4.1 


0.44 


0.9 


0.20 


Slovak Republic 


17.4 


0.84 


11.6 


0.65 


8.2 


0.66 


Spain 


15.3 


0.75 


6.0 


0.48 


1.6 


0.27 


Sweden 


18.2 


0.79 


10.0 


0.57 


4.2 


0.41 


Switzerland 


21.4 


0.89 


15.9 


0.73 


11.7 


1.09 


Turkey 


5.8 


1.01 


2.5 


0.68 


2.1 


0.94 


United States 


14.2 


0.71 


6.5 


0.51 


2.3 


0.33 


Non-OECD countries 


Hong Kong-China 


21.5 


1.15 


19.9 


0.95 


15.6 


1.05 


Indonesia 


1.8 


0.38 


0.4 


0.14 


0.1 


0.05 


Latvia 


16.8 


0.93 


8.2 


0.71 


3.5 


0.51 


Liechtenstein 


23.2 


4.21 


16.5 


2.59 


10.1 


1.78 


Macao-China 


23.2 


1.75 


13.7 


1.27 


7.2 


0.92 


Russian Federation 


14.2 


0.86 


7.7 


0.74 


4.3 


0.63 


Serbia and Montenegro 


8.6 


0.92 


2.8 


0.46 


0.9 


0.22 


Thailand 


7.0 


0.62 


2.2 


0.37 


0.5 


0.18 


Tunisia 


2.1 


0.37 


0.5 


0.14 


# 


t 


Uruguay 


6.7 


0.50 


2.2 


0.41 


0.4 


0.14 


United Kingdom 1 


17.9 


0.61 


9.7 


0.65 


3.9 


0.40 



t Not applicable. 

# Rounds to zero. 

'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a 
majority of items at that level. Students were classified into mathematics literacy levels according to their 
scores. Exact cut point scores are as follows: below level 1 (a score less than or equal to 357.77)', level 1 (a 
score greater than 357.77 and less than or equal to 420.07)\ level 2 (a score greater than 420.07 and less than or 
equal to 482.38)\ level 3 (a score greater than 482.38 and less than or equal to 544.68)\ level 4 (a score greater 
than 544.68 and less than or equal to 606.99)\ level 5 (a score greater than 606.99 and less than or equal to 
669. 3); level 6 (a score greater than 669.3). The OECD average is the average of the national averages of the 
OECD member countries with data available. Because PISA is principally an OECD study, the results for 
non-OECD countries are displayed separately from those of the OECD countries and are not included in 
the OECD average, s.e. means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-9. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the mathematics literacy change and relationships subscale, by country: 2003 



Country 


Below level 1 




Level 1 




Level 2 




Level 3 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


10.2 


0.19 


13.0 


0.15 


19.8 


0.14 


22.0 


0.19 


OECD countries 


Australia 


4.8 


0.44 


9.5 


0.53 


18.5 


0.59 


23.8 


0.75 


Austria 


8.6 


0.84 


14.1 


0.95 


20.5 


0.94 


22.5 


1.10 


Belgium 


7.6 


0.60 


9.7 


0.59 


14.8 


0.63 


18.2 


0.74 


Canada 


2.9 


0.25 


7.6 


0.36 


17.2 


0.57 


24.9 


0.54 


Czech Republic 


5.7 


0.74 


11.8 


0.96 


20.8 


0.88 


23.5 


0.82 


Denmark 


6.3 


0.61 


11.9 


0.82 


20.4 


1.06 


24.5 


0.85 


Finland 


2.7 


0.32 


7.0 


0.56 


16.1 


0.71 


24.5 


0.94 


France 


6.4 


0.76 


9.5 


0.71 


18.2 


0.68 


23.9 


0.91 


Germany 


9.5 


0.87 


12.6 


0.74 


18.5 


0.90 


20.6 


0.76 


Greece 


23.3 


1.36 


19.9 


0.94 


22.9 


0.83 


18.0 


0.89 


Hungary 


8.4 


0.81 


14.5 


0.72 


22.0 


1.16 


23.5 


0.98 


Iceland 


6.3 


0.44 


12.0 


0.60 


20.2 


0.77 


24.4 


0.79 


Ireland 


5.1 


0.51 


11.2 


0.86 


22.6 


0.84 


27.0 


1.07 


Italy 


18.2 


1.25 


19.2 


0.84 


23.7 


0.78 


20.4 


0.86 


Japan 


6.4 


0.73 


8.5 


0.72 


15.7 


0.78 


20.6 


0.85 


Korea, Republic of 


3.0 


0.40 


7.0 


0.74 


15.7 


0.97 


22.3 


0.93 


Luxembourg 


10.7 


0.60 


15.3 


0.91 


21.5 


1.07 


22.5 


0.88 


Mexico 


47.2 


1.74 


24.1 


0.85 


17.0 


0.88 


8.6 


0.81 


Netherlands 


1.4 


0.37 


7.2 


0.82 


16.4 


1.22 


22.7 


1.13 


New Zealand 


5.6 


0.60 


10.2 


0.87 


17.5 


0.69 


22.5 


0.98 


Norway 


9.5 


0.70 


15.1 


0.70 


22.8 


1.05 


23.9 


0.78 


Poland 


10.1 


0.78 


16.1 


0.71 


23.6 


0.81 


23.0 


0.91 


Portugal 


13.6 


1.28 


17.5 


1.04 


23.8 


0.93 


22.5 


1.05 


Slovak Republic 


9.7 


0.91 


14.3 


0.86 


21.0 


0.92 


22.4 


0.92 


Spain 


11.3 


0.67 


14.9 


0.96 


22.9 


0.71 


24.0 


0.93 


Sweden 


9.4 


0.64 


12.6 


0.64 


19.6 


0.87 


21.7 


0.87 


Switzerland 


7.6 


0.57 


10.1 


0.58 


17.3 


1.05 


21.3 


0.99 


Turkey 


30.0 


2.04 


21.1 


1.06 


20.1 


1.16 


13.9 


1.23 


United States 


10.4 


0.84 


14.4 


0.65 


22.6 


0.78 


24.3 


0.71 


Non-OECD countries 


Hong Kong-China 


5.6 


0.85 


8.0 


0.78 


14.5 


1.08 


20.6 


1.01 


Indonesia 


59.6 


1.78 


20.2 


0.82 


12.3 


0.81 


5.4 


0.63 


Latvia 


10.6 


0.98 


14.7 


1.08 


22.2 


1.29 


23.5 


1.24 


Liechtenstein 


4.6 


1.13 


10.0 


1.89 


15.1 


2.43 


20.7 


3.01 


Macao-China 


5.2 


1.13 


12.2 


1.32 


18.2 


1.52 


23.4 


1.84 


Russian Federation 


11.8 


1.14 


16.2 


0.86 


23.7 


1.02 


23.5 


0.89 


Serbia and Montenegro 


26.5 


1.60 


24.1 


1.13 


23.5 


0.93 


15.7 


0.92 


Thailand 


31.9 


1.61 


26.4 


1.29 


22.0 


0.90 


12.1 


0.84 


Tunisia 


58.8 


1.19 


20.4 


0.70 


12.9 


0.67 


5.8 


0.44 


Uruguay 


29.8 


1.31 


19.1 


0.79 


21.6 


1.10 


16.5 


1.03 


United Kingdom 1 


5.7 


0.57 


11.2 


0.85 


20.3 


0.89 


24.6 


0.85 



See notes at end of table. 



PISA 2003 Results From the U.S. Perspective 



Table B-9. Percentage distribution of 15-year-old students scoring at 
each proficiency level on the mathematics literacy change 
and relationships subscale, by country: 2003 — Continued 



Country 


Level 4 




Level 5 




Level 6 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


18.5 


0.20 


11.1 


0.13 


5.3 


0.11 


OECD countries 


Australia 


22.9 


0.65 


14.0 


0.65 


6.5 


0.57 


Austria 


18.8 


1.03 


10.9 


0.81 


4.6 


0.49 


Belgium 


19.7 


0.71 


17.5 


0.86 


12.4 


0.52 


Canada 


24.4 


0.56 


15.6 


0.58 


7.3 


0.44 


Czech Republic 


19.4 


0.83 


12.5 


0.71 


6.4 


0.61 


Denmark 


20.7 


0.84 


11.4 


0.78 


4.6 


0.48 


Finland 


24.1 


0.80 


16.7 


0.66 


8.9 


0.53 


France 


22.2 


0.81 


14.2 


0.69 


5.6 


0.47 


Germany 


19.6 


0.86 


13.2 


0.76 


6.1 


0.52 


Greece 


10.8 


0.95 


4.0 


0.46 


1.1 


0.19 


Hungary 


18.4 


0.83 


9.6 


0.74 


3.6 


0.42 


Iceland 


21.0 


0.80 


11.9 


0.67 


4.2 


0.41 


Ireland 


21.6 


0.85 


10.2 


0.63 


2.3 


0.35 


Italy 


11.8 


0.84 


5.2 


0.42 


1.5 


0.19 


Japan 


21.1 


1.10 


16.4 


0.79 


11.3 


1.18 


Korea, Republic of 


23.6 


0.99 


17.5 


0.91 


10.9 


1.07 


Luxembourg 


18.1 


1.00 


8.5 


0.61 


3.4 


0.36 


Mexico 


2.6 


0.39 


0.4 


0.13 


0.1 


0.03 


Netherlands 


21.8 


1.14 


19.2 


0.89 


11.3 


0.71 


New Zealand 


22.2 


0.79 


14.0 


0.68 


7.9 


0.46 


Norway 


17.4 


0.89 


8.3 


0.61 


2.9 


0.39 


Poland 


16.1 


0.84 


7.9 


0.59 


3.3 


0.33 


Portugal 


15.1 


0.91 


5.8 


0.47 


1.7 


0.25 


Slovak Republic 


18.1 


0.96 


10.1 


0.74 


4.4 


0.46 


Spain 


17.1 


0.62 


7.7 


0.54 


2.0 


0.24 


Sweden 


18.3 


0.84 


11.6 


0.53 


6.7 


0.63 


Switzerland 


20.9 


0.76 


13.9 


0.82 


8.8 


0.91 


Turkey 


7.9 


1.15 


3.8 


0.80 


3.2 


1.17 


United States 


17.7 


0.82 


8.4 


0.62 


2.2 


0.31 


Non-OECD countries 


Hong Kong-China 


23.0 


1.00 


18.6 


0.98 


9.8 


0.89 


Indonesia 


1.9 


0.38 


0.6 


0.19 


0.1 


0.08 


Latvia 


17.6 


1.16 


8.2 


0.73 


3.2 


0.50 


Liechtenstein 


20.5 


3.39 


18.6 


2.26 


10.5 


1.65 


Macao-China 


21.6 


1.81 


13.8 


1.17 


5.7 


0.97 


Russian Federation 


15.3 


1.09 


6.9 


0.68 


2.6 


0.41 


Serbia and Montenegro 


7.2 


0.68 


2.5 


0.40 


0.5 


0.13 


Thailand 


5.3 


0.58 


1.8 


0.38 


0.4 


0.16 


Tunisia 


1.8 


0.28 


0.4 


0.14 


# 


t 


Uruguay 


8.8 


0.75 


3.4 


0.40 


0.9 


0.23 


United Kingdom 1 


21.4 


0.73 


11.7 


0.72 


4.9 


0.46 



t Not applicable. 

# Rounds to zero. 

'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a 
majority of items at that level. Students were classified into mathematics literacy levels according to their 
scores. Exact cut point scores are as follows: below level 1 (a score less than or equal to 357.77)\ level 1 (a 
score greater than 357.77 and less than or equal to 420.07 ); level 2 (a score greater than 420.07 and less than or 
equal to 482.38)\ level 3 (a score greater than 482.38 and less than or equal to 544.68)\ level 4 (a score greater 
than 544.68 and less than or equal to 606.99 ); level 5 (a score greater than 606.99 and less than or equal to 
669. 3); level 6 (a score greater than 669.3). The OECD average is the average of the national averages of the 
OECD member countries with data available. Because PISA is principally an OECD study, the results for 
non-OECD countries are displayed separately from those of the OECD countries and are not included in 
the OECD average, s.e. means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-10. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the mathematics literacy uncertainty subscale, by country: 2003 



Country 


Below level 1 




Level 1 




Level 2 




Level 3 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


7.4 


0.15 


13.3 


0.17 


21.5 


0.16 


23.8 


0.17 


OECD countries 


Australia 


4.1 


0.40 


9.0 


0.47 


17.5 


0.57 


23.8 


0.59 


Austria 


7.4 


0.74 


15.2 


0.98 


22.9 


1.27 


24.3 


1.11 


Belgium 


6.2 


0.54 


11.1 


0.54 


17.3 


0.56 


20.4 


0.62 


Canada 


2.0 


0.22 


6.4 


0.37 


16.5 


0.55 


25.6 


0.50 


Czech Republic 


5.2 


0.62 


14.4 


0.85 


24.4 


1.09 


24.2 


0.96 


Denmark 


4.4 


0.59 


10.4 


0.69 


20.8 


0.78 


25.8 


0.84 


Finland 


1.6 


0.23 


5.5 


0.64 


15.4 


0.65 


27.2 


0.80 


France 


6.0 


0.68 


12.3 


0.87 


20.9 


0.79 


25.3 


1.03 


Germany 


8.7 


0.80 


15.2 


0.78 


21.8 


0.92 


22.6 


1.01 


Greece 


12.8 


1.13 


20.4 


1.30 


27.3 


1.00 


23.1 


0.91 


Hungary 


6.0 


0.73 


15.2 


0.93 


26.2 


1.06 


26.5 


0.90 


Iceland 


4.0 


0.38 


8.9 


0.59 


18.8 


0.69 


24.4 


1.07 


Ireland 


3.6 


0.45 


10.2 


0.74 


21.2 


0.86 


26.5 


0.93 


Italy 


13.7 


1.05 


18.9 


0.70 


25.6 


0.71 


22.2 


0.88 


Japan 


4.9 


0.61 


9.1 


0.91 


17.5 


0.83 


23.7 


1.08 


Korea, Republic of 


2.2 


0.27 


7.2 


0.63 


17.3 


0.83 


25.0 


0.96 


Luxembourg 


8.2 


0.41 


14.6 


0.76 


22.8 


1.05 


24.5 


1.21 


Mexico 


35.3 


1.66 


30.6 


1.29 


21.3 


1.02 


9.5 


0.84 


Netherlands 


1.0 


0.24 


6.7 


0.83 


17.0 


1.00 


23.4 


1.25 


New Zealand 


3.9 


0.55 


9.4 


0.77 


18.0 


1.04 


23.3 


1.01 


Norway 


5.7 


0.56 


11.8 


0.79 


20.6 


0.82 


24.4 


1.22 


Poland 


5.2 


0.63 


13.9 


0.88 


25.7 


1.04 


27.4 


0.86 


Portugal 


9.0 


1.09 


18.4 


1.08 


27.7 


1.01 


25.6 


1.11 


Slovak Republic 


8.6 


0.99 


17.9 


0.84 


26.8 


0.92 


24.1 


0.93 


Spain 


7.1 


0.64 


13.7 


0.69 


25.5 


0.80 


26.9 


0.78 


Sweden 


6.4 


0.54 


11.8 


0.72 


21.5 


0.82 


22.9 


0.79 


Switzerland 


6.3 


0.49 


10.7 


0.65 


19.1 


0.78 


24.0 


0.88 


Turkey 


18.6 


1.52 


25.6 


1.37 


25.3 


1.17 


16.6 


1.30 


United States 


9.0 


0.76 


14.9 


0.71 


22.3 


0.74 


23.6 


0.75 


Non-OECD countries 


Hong Kong-China 


3.3 


0.65 


6.3 


0.70 


12.5 


0.86 


19.3 


0.89 


Indonesia 


35.3 


1.57 


36.7 


1.01 


20.4 


1.08 


6.2 


0.69 


Latvia 


8.3 


0.76 


17.8 


1.20 


28.1 


1.25 


25.7 


1.20 


Liechtenstein 


5.2 


1.60 


9.5 


1.95 


18.4 


2.33 


23.0 


2.90 


Macao-China 


2.5 


0.57 


7.2 


1.29 


18.9 


1.62 


27.4 


2.03 


Russian Federation 


19.0 


1.40 


24.8 


1.05 


26.3 


0.99 


18.1 


1.04 


Serbia and Montenegro 


20.1 


1.30 


27.3 


1.12 


26.8 


1.06 


17.4 


1.29 


Thailand 


18.1 


1.08 


32.8 


0.95 


29.6 


1.04 


14.1 


0.86 


Tunisia 


47.9 


1.27 


32.3 


0.96 


14.8 


0.88 


4.2 


0.61 


Uruguay 


27.1 


1.27 


23.5 


1.13 


23.5 


1.26 


16.0 


0.77 


United Kingdom 1 


3.8 


0.42 


10.1 


0.64 


20.4 


0.69 


25.7 


0.76 



See notes at end of table. 
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Table B-10. Percentage distribution of 15-year-old students scoring at 

each proficiency level on the mathematics literacy uncertainty 
subscale, by country: 2003 — Continued 



Country 


Level 4 




Level 5 




Level 6 




Percent 


s . e . 


Percent 


s . e . 


Percent 


s . e . 


OECD average 


19.2 


0.17 


10.6 


0.14 


4.2 


0.10 


OECD countries 


Australia 


23.0 


0.60 


15.1 


0.52 


7.4 


0.52 


Austria 


17.9 


1.14 


9.3 


0.71 


3.0 


0.41 


Belgium 


20.8 


0.57 


15.8 


0.53 


8.4 


0.43 


Canada 


26.3 


0.56 


16.4 


0.55 


6.8 


0.47 


Czech Republic 


19.2 


0.87 


9.3 


0.85 


3.3 


0.44 


Denmark 


22.0 


0.82 


12.6 


0.66 


4.0 


0.43 


Finland 


27.0 


0.87 


16.4 


0.82 


6.8 


0.57 


France 


21.7 


0.75 


11.0 


0.65 


2.8 


0.28 


Germany 


19.0 


0.94 


9.7 


0.78 


2.9 


0.32 


Greece 


11.8 


0.94 


4.0 


0.59 


0.7 


0.15 


Hungary 


17.3 


0.91 


7.1 


0.70 


1.6 


0.32 


Iceland 


22.9 


0.86 


14.8 


0.68 


6.1 


0.50 


Ireland 


22.0 


0.93 


12.4 


0.72 


4.0 


0.39 


Italy 


13.0 


0.79 


5.1 


0.38 


1.4 


0.16 


Japan 


23.5 


1.35 


14.8 


0.95 


6.6 


0.95 


Korea, Republic of 


25.7 


0.95 


15.7 


0.81 


6.7 


0.80 


Luxembourg 


18.2 


0.68 


8.7 


0.60 


2.9 


0.41 


Mexico 


2.7 


0.43 


0.5 


0.11 


# 


t 


Netherlands 


23.2 


1.30 


19.1 


1.06 


9.5 


0.78 


New Zealand 


22.1 


0.97 


14.6 


0.73 


8.6 


0.54 


Norway 


20.3 


0.84 


11.6 


0.89 


5.6 


0.42 


Poland 


18.7 


1.04 


7.5 


0.75 


1.6 


0.25 


Portugal 


14.5 


1.00 


4.2 


0.38 


0.6 


0.18 


Slovak Republic 


15.7 


0.80 


5.6 


0.46 


1.2 


0.23 


Spain 


18.4 


0.71 


6.9 


0.51 


1.5 


0.26 


Sweden 


19.7 


0.76 


12.1 


0.61 


5.6 


0.54 


Switzerland 


21.2 


0.83 


12.9 


0.98 


5.8 


0.74 


Turkey 


8.0 


1.14 


3.4 


0.77 


2.6 


1.07 


United States 


17.4 


0.75 


9.5 


0.70 


3.2 


0.39 


Non-OECD countries 


Hong Kong-China 


24.8 


1.19 


21.1 


1.10 


12.7 


1.12 


Indonesia 


1.3 


0.33 


0.1 


0.05 


# 


t 


Latvia 


14.6 


0.92 


4.5 


0.52 


1.0 


0.25 


Liechtenstein 


23.8 


2.97 


14.9 


2.54 


5.1 


1.37 


Macao-China 


23.5 


1.70 


14.9 


1.53 


5.4 


1.02 


Russian Federation 


8.6 


0.76 


2.7 


0.43 


0.5 


0.14 


Serbia and Montenegro 


6.7 


0.72 


1.5 


0.28 


0.2 


0.14 


Thailand 


4.3 


0.50 


1.1 


0.25 


0.1 


0.06 


Tunisia 


0.8 


0.28 


# 


t 


# 


t 


Uruguay 


7.1 


0.54 


2.4 


0.29 


0.4 


0.13 


United Kingdom 1 


22.3 


0.65 


12.8 


0.71 


4.8 


0.51 



f Not applicable. 

# Rounds to zero. 

'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a 
majority of items at that level. Students were classified into mathematics literacy levels according to their 
scores. Exact cut point scores are as follows: below level 1 (a score less than or equal to 357.77)', level 1 (a 
score greater than 357.77 and less than or equal to 420.07)\ level 2 (a score greater than 420.07 and less than or 
equal to 482.38)\ level 3 (a score greater than 482.38 and less than or equal to 544.68)\ level 4 (a score greater 
than 544.68 and less than or equal to 606.99)\ level 5 (a score greater than 606.99 and less than or equal to 
669. 3); level 6 (a score greater than 669.3). The OECD average is the average of the national averages of the 
OECD member countries with data available. Because PISA is principally an OECD study, the results for 
non-OECD countries are displayed separately from those of the OECD countries and are not included in 
the OECD average, s.e. means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 
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Table B-11. Average selected mathematics literacy subscale scores of 15-year-old students, by 
country: 2000 and 2003 







Space and shape 




Change and relationships 




Country 


2000 




2003 




2000 




2003 






Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


OECD average 


— 


— 


498.8 


0.70 


— 


— 


496.3 


0.65 


OECD countries 


Australia 


519.7 


3.14 


520.5 


2.33 


522.2 


3.21 


525.3 


2.30 


Austria 


510.1 


2.85 


515.2 


3.48 


499.3 


3.10 


499.7 


3.60 


Belgium 


501.9 


3.05 


529.6 


2.26 


513.7 


3.82 


535.3 


2.45 


Canada 


515.3 


1.50 


517.8 


1.81 


519.7 


1.26 


536.7 


1.93 


Czech Republic 


510.1 


3.51 


528.0 


4.17 


484.4 


2.98 


514.1 


3.55 


Denmark 


526.2 


2.58 


512.4 


2.76 


499.3 


2.70 


509.3 


2.99 


Finland 


533.0 


2.05 


539.0 


2.04 


529.5 


2.12 


543.0 


2.19 


France 


501.0 


2.69 


507.6 


2.98 


514.5 


2.70 


519.7 


2.62 


Germany 


485.9 


3.10 


499.6 


3.28 


485.1 


2.41 


507.2 


3.73 


Greece 


450.4 


4.35 


437.1 


3.80 


429.9 


5.21 


435.6 


4.31 


Hungary 


478.4 


3.30 


479.0 


3.34 


478.6 


4.05 


494.6 


3.10 


Iceland 


518.8 


2.30 


503.5 


1.46 


507.5 


2.79 


509.4 


1.43 


Ireland 


473.5 


3.23 


476.2 


2.43 


501.1 


2.74 


505.9 


2.45 


Italy 


454.7 


3.61 


470.3 


3.15 


442.8 


3.00 


452.1 


3.21 


Japan 


564.8 


5.06 


553.1 


4.31 


536.2 


5.12 


536.1 


4.33 


Korea, Republic of 


538.2 


3.59 


551.7 


3.80 


530.1 


2.55 


547.5 


3.52 


Luxembourg 


448.6 


3.02 


488.2 


1.35 


423.8 


2.63 


487.0 


1.15 


Mexico 


399.8 


2.62 


381.7 


3.20 


357.6 


3.14 


364.1 


4.14 


New Zealand 


523.7 


4.01 


524.9 


2.34 


527.1 


3.04 


525.7 


2.37 


Norway 


490.5 


3.09 


482.6 


2.55 


493.6 


3.05 


487.7 


2.64 


Poland 


469.9 


5.49 


490.3 


2.66 


451.2 


5.66 


484.3 


2.70 


Portugal 


439.5 


3.54 


450.2 


3.43 


448.5 


3.56 


467.8 


3.96 


Spain 


472.7 


2.57 


476.5 


2.59 


467.8 


2.81 


480.7 


2.80 


Sweden 


509.8 


2.64 


498.3 


2.57 


501.6 


2.63 


505.0 


2.94 


Switzerland 


538.9 


3.58 


539.5 


3.50 


509.6 


4.84 


522.7 


3.65 


United States 


461.2 


4.94 


472.0 


2.78 


485.7 


6.02 


485.5 


3.03 


Non-OECD countries 


Latvia 


— 


— 


486.4 


4.04 


— 


— 


487.2 


4.36 


Liechtenstein 


— 


— 


538.2 


4.58 


— 


— 


539.5 


3.67 


Russian Federation 


— 


— 


474.3 


4.69 


— 


— 


476.8 


4.64 


United Kingdom 1 


504.9 


2.58 


496.0 


2.50 


519.2 


2.21 


512.9 


2.54 



— Not available. 

'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the national averages of the OECD member countries with data available. Because 
PISA is principally an OECD study, the results for non-OECD countries are displayed separately from those of the OECD countries 
and are not included in the OECD average, s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2000 and 2003. 
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Table B-12. Average problem-solving 
scores of 15-year-old 
students, by country: 2003 



Country 


Average 


s.e. 


OECD average 


500.0 


0.65 


OECD countries 


Australia 


529.9 


1.98 


Austria 


506.1 


3.18 


Belgium 


524.7 


2.21 


Canada 


529.3 


1.74 


Czech Republic 


516.4 


3.42 


Denmark 


516.8 


2.54 


Finland 


547.6 


1.86 


France 


519.2 


2.67 


Germany 


513.5 


3.24 


Greece 


448.5 


3.97 


Hungary 


501.1 


2.86 


Iceland 


504.7 


1.38 


Ireland 


498.5 


2.34 


Italy 


469.5 


3.10 


Japan 


547.3 


4.05 


Korea, Republic of 


550.4 


3.06 


Luxembourg 


493.7 


1.37 


Mexico 


384.4 


4.29 


Netherlands 


520.2 


2.95 


New Zealand 


532.8 


2.17 


Norway 


489.8 


2.60 


Poland 


486.6 


2.78 


Portugal 


469.9 


3.87 


Slovak Republic 


491.8 


3.38 


Spain 


482.3 


2.73 


Sweden 


508.6 


2.44 


Switzerland 


521.3 


3.05 


Turkey 


407.6 


6.03 


United States 


477.4 


3.13 


Non-OECD countries 


Hong Kong-China 


547.9 


4.18 


Indonesia 


361.5 


3.29 


Latvia 


482.5 


3.90 


Liechtenstein 


529.5 


3.95 


Macao-China 


532.4 


2.53 


Russian Federation 


478.6 


4.59 


Serbia and Montenegro 


420.2 


3.32 


Thailand 


425.0 


2.72 


Tunisia 


344.8 


2.11 


Uruguay 


410.7 


3.68 


United Kingdom 1 


510.2 


2.38 



'Due to low response rates, data for the United 
Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the 
national averages of the OECD member countries 
with data available. Because PISA is principally an 
OECD study, the results for non-OECD countries are 
displayed separately from those of the OECD coun- 
tries and are not included in the OECD average, s.e. 
means standard error. 

SOURCE: Organization for Economic Cooperation and 
Development (OECD), Program for International 
Student Assessment (PISA), 2003. 
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Table B-13. Problem-solving scores of 15-year-old students, by percentiles and country: 2003 



Country 


5 th percentile 


10 th percentile 


25 th percentile 


50 th percentile 


Score 


s.e. 


Score 


s.e. 


Score 


s.e. 


Score 


s.e. 


OECD average 


328.2 


1.61 


368.4 


1.32 


434.1 


1.10 


503.1 


0.28 


OECD countries 


Australia 


370.9 


4.07 


409.3 


3.46 


469.5 


2.78 


535.1 


0.70 


Austria 


356.9 


5.14 


388.0 


4.50 


443.0 


4.08 


508.9 


1.30 


Belgium 


339.4 


5.72 


381.1 


4.66 


455.0 


3.32 


533.6 


1.19 


Canada 


378.8 


2.43 


414.0 


2.78 


470.8 


2.46 


532.1 


0.61 


Czech Republic 


356.5 


8.56 


394.0 


6.17 


454.0 


4.43 


519.0 


0.94 


Denmark 


368.9 


5.02 


402.3 


4.28 


458.5 


3.06 


520.0 


1.14 


Finland 


408.6 


4.67 


441.6 


2.80 


494.5 


2.48 


550.5 


1.10 


France 


358.4 


6.09 


395.9 


4.78 


458.6 


3.94 


523.8 


1.65 


Germany 


350.7 


5.91 


383.5 


5.35 


446.9 


4.78 


520.5 


1.20 


Greece 


283.1 


5.57 


319.2 


5.25 


382.7 


4.52 


450.8 


2.25 


Hungary 


343.0 


5.82 


377.6 


4.05 


436.5 


3.76 


503.9 


1.80 


Iceland 


358.0 


5.47 


393.4 


3.28 


449.8 


2.22 


510.1 


1.44 


Ireland 


364.0 


4.48 


394.6 


3.80 


444.9 


3.14 


500.0 


1.06 


Italy 


288.8 


8.75 


334.2 


6.47 


406.1 


4.72 


475.2 


1.53 


Japan 


362.1 


8.27 


405.8 


6.83 


481.4 


5.71 


555.8 


1.56 


Korea, Republic of 


404.1 


4.63 


437.9 


5.21 


493.9 


3.85 


553.7 


1.52 


Luxembourg 


339.1 


3.70 


373.4 


2.32 


432.1 


2.44 


496.0 


1.54 


Mexico 


226.6 


5.44 


261.8 


5.18 


317.2 


5.20 


384.5 


1.93 


Netherlands 


372.2 


5.89 


401.5 


5.14 


456.1 


4.89 


521.3 


1.73 


New Zealand 


369.8 


3.75 


405.6 


4.23 


467.8 


3.65 


538.0 


1.21 


Norway 


322.3 


5.54 


360.9 


4.64 


424.4 


3.71 


493.6 


2.01 


Poland 


338.0 


5.55 


372.0 


4.07 


428.4 


3.10 


487.8 


1.44 


Portugal 


311.2 


7.91 


345.3 


6.84 


408.6 


5.65 


475.2 


2.18 


Slovak Republic 


336.6 


7.13 


370.4 


5.86 


430.1 


4.71 


494.0 


0.79 


Spain 


321.8 


4.78 


361.2 


4.12 


420.9 


3.55 


486.0 


0.93 


Sweden 


359.8 


6.39 


395.2 


4.41 


451.3 


3.04 


511.4 


1.38 


Switzerland 


358.2 


5.73 


397.0 


4.04 


461.3 


3.32 


526.8 


1.87 


Turkey 


257.2 


7.84 


290.7 


6.58 


343.0 


5.16 


402.3 


1.34 


United States 


312.2 


5.63 


347.3 


4.55 


409.7 


4.07 


479.5 


1.71 


Non-OECD countries 


Hong Kong-China 


376.2 


10.52 


420.2 


7.86 


487.0 


6.12 


557.8 


1.64 


Indonesia 


244.7 


4.22 


269.9 


3.77 


311.7 


3.60 


359.3 


1.91 


Latvia 


326.2 


6.97 


361.9 


6.00 


419.9 


5.38 


484.6 


1.04 


Liechtenstein 


369.0 


14.93 


404.1 


11.09 


467.6 


6.03 


532.8 


4.89 


Macao-China 


394.8 


6.41 


424.8 


5.58 


477.8 


3.72 


535.7 


1.69 


Russian Federation 


314.2 


7.73 


351.0 


6.99 


412.7 


5.72 


480.7 


1.69 


Serbia and Montenegro 


278.7 


4.15 


310.6 


4.40 


362.8 


3.94 


420.2 


2.53 


Thailand 


293.0 


3.87 


322.4 


3.37 


369.1 


2.64 


422.0 


2.15 


Tunisia 


213.0 


4.30 


242.7 


3.10 


290.7 


2.54 


344.4 


1.01 


Uruguay 


223.8 


5.69 


265.0 


5.10 


334.5 


4.66 


413.5 


1.69 


United Kingdom 1 


352.7 


4.40 


387.1 


3.59 


446.1 


2.90 


513.2 


1.03 



See notes at end of table. 
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Table B-13. Problem-solving scores of 15-year-old students, by percentiles 
and country: 2003 — Continued 



Country 


75 th percentile 


90 th percentile 


95 th percentile 


Score 


s.e. 


Score 


s.e. 


Score 


s.e. 


OECD average 


570.9 


0.82 


625.0 


0.75 


655.6 


0.82 


OECD countries 


Australia 


593.9 


2.13 


643.5 


2.67 


671.7 


3.40 


Austria 


569.4 


3.98 


620.7 


4.18 


650.9 


4.63 


Belgium 


601.2 


2.24 


653.4 


2.21 


682.0 


2.77 


Canada 


591.1 


1.89 


640.4 


2.13 


668.8 


2.40 


Czech Republic 


582.2 


3.58 


633.7 


3.87 


663.5 


4.01 


Denmark 


578.4 


2.76 


627.0 


3.44 


654.9 


3.71 


Finland 


604.0 


2.28 


649.6 


2.33 


676.5 


3.58 


France 


585.8 


3.05 


634.7 


3.66 


662.3 


4.51 


Germany 


583.1 


4.28 


631.6 


2.66 


658.4 


3.19 


Greece 


516.6 


4.58 


574.5 


5.72 


607.5 


5.60 


Hungary 


566.7 


3.93 


622.1 


4.31 


653.1 


5.37 


Iceland 


564.3 


2.03 


609.1 


2.33 


634.4 


3.59 


Ireland 


555.3 


2.73 


600.7 


2.83 


624.5 


3.21 


Italy 


540.1 


2.95 


595.4 


3.41 


627.2 


3.56 


Japan 


620.9 


4.22 


675.3 


4.56 


705.3 


6.01 


Korea, Republic of 


609.9 


3.47 


658.3 


4.21 


686.0 


5.53 


Luxembourg 


558.0 


2.17 


609.8 


2.59 


639.9 


3.40 


Mexico 


451.5 


5.06 


509.3 


5.69 


541.6 


6.48 


Netherlands 


586.7 


3.59 


636.4 


3.34 


662.1 


3.71 


New Zealand 


600.9 


2.38 


652.7 


2.49 


682.4 


2.76 


Norway 


558.8 


3.27 


614.8 


4.17 


645.5 


4.38 


Poland 


547.6 


2.97 


600.2 


3.53 


631.7 


4.49 


Portugal 


534.3 


3.60 


586.3 


3.45 


614.5 


3.54 


Slovak Republic 


558.0 


3.58 


609.0 


3.84 


638.1 


4.16 


Spain 


547.3 


3.22 


599.4 


3.89 


629.4 


3.31 


Sweden 


570.8 


3.06 


619.0 


3.82 


647.2 


3.56 


Switzerland 


586.9 


3.86 


636.9 


4.60 


666.2 


5.18 


Turkey 


466.5 


7.69 


531.2 


11.94 


576.7 


18.61 


United States 


547.9 


3.29 


603.6 


3.97 


634.8 


4.18 


Non-OECD countries 


Hong Kong-China 


617.4 


3.18 


663.5 


2.93 


690.2 


3.72 


Indonesia 


408.6 


4.10 


456.8 


5.51 


486.9 


5.92 


Latvia 


547.3 


4.57 


599.0 


4.11 


628.4 


4.89 


Liechtenstein 


598.9 


9.30 


644.1 


10.46 


671.8 


12.04 


Macao-China 


590.0 


4.34 


632.9 


5.43 


658.9 


6.54 


Russian Federation 


546.0 


5.12 


603.7 


5.03 


637.2 


5.55 


Serbia and Montenegro 


477.7 


4.19 


529.9 


4.93 


559.8 


5.08 


Thailand 


478.5 


3.98 


531.7 


4.01 


565.4 


5.97 


Tunisia 


400.0 


2.76 


445.8 


4.11 


474.5 


4.98 


Uruguay 


488.4 


5.45 


552.1 


5.04 


589.3 


5.74 


United Kingdom 1 


576.5 


3.07 


628.9 


3.69 


659.2 


3.98 



'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the national averages of the OECD member countries with 
data available. Because PISA is principally an OECD study, the results for non-OECD countries are dis- 
played separately from those of the OECD countries and are not included in the OECD average, s.e. 
means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 




Table B-14. Standard deviations of 15-year-old 
students' problem-solving scores, 
by country: 2003 



Country 


Standard deviation 


s.e. 


OECD average 


100.0 


0.44 


OECD countries 


Australia 


91.4 


1.35 


Austria 


89.9 


1.71 


Belgium 


104.4 


1.57 


Canada 


88.4 


0.93 


Czech Republic 


92.8 


1.92 


Denmark 


87.3 


1.51 


Finland 


82.0 


1.15 


France 


92.9 


2.08 


Germany 


94.7 


1.75 


Greece 


98.8 


1.67 


Hungary 


94.1 


2.03 


Iceland 


84.8 


1.15 


Ireland 


79.6 


1.35 


Italy 


102.1 


2.14 


Japan 


104.9 


2.72 


Korea, Republic of 


86.4 


1.95 


Luxembourg 


91.6 


1.03 


Mexico 


96.1 


2.01 


Netherlands 


89.4 


2.03 


New Zealand 


95.7 


1.24 


Norway 


98.8 


1.65 


Poland 


90.4 


1.68 


Portugal 


92.4 


2.11 


Slovak Republic 


92.8 


2.39 


Spain 


93.6 


1.25 


Sweden 


88.4 


1.58 


Switzerland 


94.0 


1.88 


Turkey 


96.7 


4.43 


United States 


98.1 


1.29 


Non-OECD countries 


Hong Kong-China 


97.2 


2.90 


Indonesia 


73.2 


1.74 


Latvia 


92.1 


1.75 


Liechtenstein 


92.7 


4.20 


Macao-China 


81.3 


2.55 


Russian Federation 


98.5 


2.11 


Serbia and Montenegro 


85.8 


1.56 


Thailand 


82.0 


1.59 


Tunisia 


79.5 


1.42 


Uruguay 


111.7 


1.93 


United Kingdom 1 


93.2 


1.21 



'Due to low response rates, data for the United Kingdom are not 
discussed in this report. 

NOTE:The OECD average is the average of the national aver- 
ages of the OECD member countries with data available. 
Because PISA is principally an OECD study, the results for non- 
OECD countries are displayed separately from those of the 
OECD countries and are not included in the OECD average, s.e. 
means standard error. 

SOURCE: Organization for Economic Cooperation and 
Development (OECD), Program for International Student 
Assessment (PISA), 2003. 



PISA 2003 Results From the U.S. Perspective 



Table B-15. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the problem-solving scale, by country: 2003 



Country 


Below level 1 




Level 1 




Level 2 




Level 3 




Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


17.3 


0.25 


30.4 


0.20 


34.2 


0.22 


18.2 


0.18 


OECD countries 


Australia 


9.4 


0.59 


25.8 


0.67 


39.1 


0.83 


25.7 


0.84 


Austria 


13.6 


1.00 


32.3 


1.07 


36.8 


1.06 


17.2 


1.16 


Belgium 


14.0 


0.80 


23.8 


0.88 


34.1 


0.82 


28.1 


0.81 


Canada 


8.5 


0.48 


27.0 


0.70 


40.0 


0.69 


24.5 


0.72 


Czech Republic 


12.1 


1.14 


29.4 


1.20 


37.0 


1.12 


21.5 


1.24 


Denmark 


10.5 


0.76 


30.2 


0.92 


39.2 


0.90 


20.1 


0.93 


Finland 


4.6 


0.51 


22.1 


0.84 


43.3 


0.82 


30.1 


0.92 


France 


11.7 


0.95 


28.1 


1.04 


37.5 


1.05 


22.7 


0.99 


Germany 


14.2 


1.00 


27.7 


1.14 


36.4 


1.49 


21.7 


1.39 


Greece 


32.7 


1.53 


36.1 


1.01 


24.3 


1.21 


7.0 


0.80 


Hungary 


16.1 


0.98 


31.8 


1.40 


34.9 


1.20 


17.2 


1.16 


Iceland 


12.4 


0.66 


32.5 


1.01 


40.2 


0.97 


14.9 


0.64 


Ireland 


12.5 


0.91 


36.9 


1.17 


38.3 


1.05 


12.3 


0.76 


Italy 


24.7 


1.32 


34.7 


1.16 


30.0 


0.99 


10.6 


0.66 


Japan 


9.9 


1.00 


20.0 


0.98 


34.5 


1.25 


35.6 


1.58 


Korea, Republic of 


5.2 


0.54 


21.6 


1.02 


40.8 


1.11 


32.4 


1.35 


Luxembourg 


17.0 


0.71 


34.1 


0.95 


34.7 


1.04 


14.2 


0.56 


Mexico 


58.1 


1.86 


29.7 


1.07 


10.9 


0.99 


1.3 


0.24 


Netherlands 


10.7 


1.13 


30.5 


1.31 


35.8 


1.41 


23.0 


1.13 


New Zealand 


9.9 


0.81 


25.3 


0.85 


36.5 


1.01 


28.3 


0.92 


Norway 


19.4 


0.88 


32.6 


1.15 


33.1 


0.97 


14.9 


0.84 


Poland 


17.5 


0.98 


37.2 


1.02 


33.6 


1.14 


11.7 


0.66 


Portugal 


23.9 


1.73 


36.5 


1.09 


31.0 


1.36 


8.6 


0.63 


Slovak Republic 


17.5 


1.40 


34.4 


1.16 


34.0 


1.34 


14.1 


0.96 


Spain 


20.1 


0.87 


35.5 


1.08 


32.9 


1.15 


11.6 


0.83 


Sweden 


12.0 


0.85 


32.4 


1.12 


38.2 


1.02 


17.4 


0.99 


Switzerland 


11.4 


0.74 


26.8 


1.02 


38.7 


1.10 


23.1 


1.42 


Turkey 


51.2 


2.51 


32.5 


1.58 


12.4 


1.64 


3.9 


1.24 


United States 


23.7 


1.12 


33.7 


0.83 


30.3 


1.03 


12.4 


0.78 


Non-OECD countries 


Hong Kong-China 


8.0 


1.05 


20.5 


0.99 


36.5 


1.17 


35.0 


1.39 


Indonesia 


73.5 


1.71 


22.9 


1.36 


3.5 


0.64 


0.1 


0.07 


Latvia 


20.3 


1.48 


35.6 


1.27 


32.5 


1.39 


11.6 


0.96 


Liechtenstein 


10.2 


1.54 


26.0 


2.35 


36.8 


3.56 


27.1 


2.62 


Macao-China 


6.3 


0.82 


27.3 


1.37 


42.1 


2.00 


24.2 


1.59 


Russian Federation 


22.8 


1.69 


34.5 


1.03 


30.6 


1.28 


12.2 


1.02 


Serbia and Montenegro 


42.6 


1.70 


39.5 


1.16 


15.8 


1.17 


2.1 


0.31 


Thailand 


41.4 


1.57 


40.5 


1.08 


15.6 


1.10 


2.6 


0.47 


Tunisia 


77.1 


1.05 


20.4 


0.82 


2.5 


0.51 


0.1 


0.05 


Uruguay 


47.2 


1.58 


30.5 


1.33 


17.5 


1.23 


4.7 


0.54 


United Kingdom 1 


13.7 


0.75 


30.3 


1.09 


36.6 


1.00 


19.5 


0.97 



'Due to low response rates, data for the United Kingdom are not discussed in this report. 

NOTE: In order to reach a particular proficiency level, a student must have been able to correctly answer a majority of items at that 
level. Students were classified into problem solving levels according to their scores. Exact cut point scores are as follows: below 
level 1 (a score less than or equal to 404.06 ); level 1 (a score greater than 404.06 and less than or equal to 498.08)', level 2 (a score greater 
than 498.08 and less than or equal to 592. 10)\ level 3 (a score greater than 592. 10). The OECD average is the average of the national 
averages of the OECD member countries with data available. Because PISA is principally an OECD study, the results for non- 
OECD countries are displayed separately from those of the OECD countries and are not included in the OECD average, s.e. means 
standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-16. Average reading literacy scores of 15-year-old 
students, by country: 2000 and 2003 



Country 


2000 




2003 




Average 


s.e. 


Average 


s.e. 


OECD average 


500.0 


0.64 


494.2 


0.65 


OECD countries 


Australia 


528.3 


3.52 


525.4 


2.13 


Austria 


507.1 


2.40 


490.7 


3.76 


Belgium 


507.1 


3.56 


507.9 


2.57 


Canada 


534.3 


1.56 


527.9 


1.75 


Czech Republic 


491.6 


2.37 


488.5 


3.46 


Denmark 


496.9 


2.35 


492.3 


2.82 


Finland 


546.5 


2.58 


543.5 


1.64 


France 


504.7 


2.73 


496.2 


2.68 


Germany 


484.0 


2.47 


491.4 


3.39 


Greece 


473.8 


4.97 


472.3 


4.10 


Hungary 


480.0 


3.95 


481.9 


2.47 


Iceland 


506.9 


1.45 


491.7 


1.56 


Ireland 


526.7 


3.24 


515.5 


2.63 


Italy 


487.5 


2.91 


475.7 


3.04 


Japan 


522.2 


5.21 


498.1 


3.92 


Korea, Republic of 


524.8 


2.42 


534.1 


3.09 


Luxembourg 


441.3 


1.59 


479.4 


1.48 


Mexico 


422.0 


3.32 


399.7 


4.09 


Netherlands 


— 


— 


513.1 


2.85 


New Zealand 


528.8 


2.78 


521.6 


2.46 


Norway 


505.3 


2.80 


499.7 


2.78 


Poland 


479.1 


4.46 


496.6 


2.88 


Portugal 


470.2 


4.52 


477.6 


3.73 


Slovak Republic 


— 


— 


469.2 


3.12 


Spain 


492.6 


2.71 


480.5 


2.60 


Sweden 


516.3 


2.20 


514.3 


2.42 


Switzerland 


494.4 


4.25 


499.1 


3.28 


Turkey 


— 


— 


441.0 


5.79 


United States 


504.4 


7.05 


495.2 


3.22 


Non-OECD countries 


Hong Kong-China 


— 


— 


509.5 


3.69 


Indonesia 


— 


— 


381.6 


3.38 


Latvia 


458.1 


5.27 


490.6 


3.67 


Liechtenstein 


482.6 


4.12 


525.1 


3.58 


Macao-China 


— 


— 


497.6 


2.16 


Russian Federation 


461.8 


4.16 


442.2 


3.94 


Serbia and Montenegro 


— 


— 


411.7 


3.56 


Thailand 


— 


— 


419.9 


2.81 


Tunisia 


— 


— 


374.6 


2.81 


Uruguay 


— 


— 


434.1 


3.43 


United Kingdom 1 


523.4 


2.56 


507.0 


2.46 



— Not available. 

'Due to low response rates, 2003 data for the United Kingdom are not discussed in 
this report. 

NOTE:The OECD average is the average of the national averages of the OECD 
member countries with data available. Because PISA is principally an OECD 
study, the results for non-OECD countries are displayed separately from those 
of the OECD countries and are not included in the OECD average, s.e. means 
standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), 
Program for International Student Assessment (PISA), 2000 and 2003. 



PISA 2003 Results From the U.S. Perspective 



Table B-17. Average science literacy scores of 15-year-old 
students, by country: 2000 and 2003 



Country 


2000 




2003 




Average 


s.e. 


Average 


s.e. 


OECD average 


500.0 


0.65 


499.6 


0.60 


OECD countries 


Australia 


527.5 


3.47 


525.1 


2.10 


Austria 


518.6 


2.55 


491.0 


3.44 


Belgium 


495.7 


4.29 


508.8 


2.44 


Canada 


529.4 


1.57 


518.7 


2.02 


Czech Republic 


511.4 


2.43 


523.3 


3.38 


Denmark 


481.0 


2.81 


475.2 


2.97 


Finland 


537.7 


2.48 


548.2 


1.92 


France 


500.5 


3.18 


511.2 


2.99 


Germany 


487.1 


2.43 


502.3 


3.64 


Greece 


460.6 


4.89 


481.0 


3.82 


Hungary 


496.1 


4.17 


503.3 


2.77 


Iceland 


495.9 


2.17 


494.7 


1.47 


Ireland 


513.4 


3.18 


505.4 


2.69 


Italy 


477.6 


3.05 


486.5 


3.13 


Japan 


550.4 


5.48 


547.6 


4.14 


Korea, Republic of 


552.1 


2.69 


538.4 


3.54 


Luxembourg 


443.1 


2.32 


482.8 


1.50 


Mexico 


421.5 


3.18 


404.9 


3.49 


Netherlands 


— 


— 


524.4 


3.15 


New Zealand 


527.7 


2.40 


520.9 


2.35 


Norway 


500.3 


2.75 


484.2 


2.87 


Poland 


483.1 


5.12 


497.8 


2.86 


Portugal 


459.0 


4.00 


467.7 


3.46 


Slovak Republic 


— 


— 


494.9 


3.71 


Spain 


490.9 


2.95 


487.1 


2.61 


Sweden 


512.1 


2.51 


506.1 


2.72 


Switzerland 


495.7 


4.45 


513.0 


3.69 


Turkey 


— 


— 


434.2 


5.89 


United States 


499.5 


7.31 


491.3 


3.08 


Non-OECD countries 


Hong Kong-China 


— 


— 


539.5 


4.26 


Indonesia 


— 


— 


395.0 


3.21 


Latvia 


460.1 


5.62 


489.1 


3.89 


Liechtenstein 


476.1 


7.09 


525.2 


4.33 


Macao-China 


— 


— 


524.7 


3.03 


Russian Federation 


460.3 


4.74 


489.3 


4.14 


Serbia and Montenegro 


— 


— 


436.4 


3.50 


Thailand 


— 


— 


429.1 


2.70 


Tunisia 


— 


— 


384.7 


2.56 


Uruguay 


— 


— 


438.4 


2.90 


United Kingdom 1 


532.0 


2.69 


518.4 


2.52 



— Not available. 

'Due to low response rates, 2003 data for the United Kingdom are not discussed in 
this report. 

NOTE:The OECD average is the average of the national averages of the OECD 
member countries with data available. Because PISA is principally an OECD 
study, the results for non-OECD countries are displayed separately from those 
of the OECD countries and are not included in the OECD average, s.e. means 
standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), 
Program for International Student Assessment (PISA), 2000 and 2003. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-18. Average combined mathematics literacy scores of 15-year-old 
students, by sex and country: 2003 



Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


505.5 


0.76 


494.4 


0.76 


11.1 


0.81 


OECD countries 


Australia 


526.9 


3.01 


521.5 


2.69 


5.3 


3.75 


Austria 


509.4 


3.95 


501.8 


3.96 


7.6 


4.41 


Belgium 


532.9 


3.40 


525.4 


3.23 


7.5 


4.81 


Canada 


540.8 


2.05 


529.6 


1.88 


11.2 


2.13 


Czech Republic 


523.8 


4.27 


508.9 


4.37 


15.00 


5.09 


Denmark 


522.7 


3.38 


506.2 


2.98 


16.6 


3.21 


Finland 


548.0 


2.46 


540.6 


2.11 


7.4 


2.67 


France 


515.3 


3.55 


506.7 


2.92 


8.5 


4.16 


Germany 


507.9 


3.98 


498.9 


3.93 


9.0 


4.37 


Greece 


455.0 


4.75 


435.6 


3.85 


19.4 


3.63 


Hungary 


493.7 


3.33 


485.9 


3.31 


7.8 


3.54 


Iceland 


507.7 


2.28 


523.1 


2.17 


-15.4 


3.46 


Ireland 


510.2 


3.01 


495.4 


3.39 


14.8 


4.19 


Italy 


474.9 


4.56 


457.1 


3.84 


17.8 


5.89 


Japan 


538.5 


5.81 


530.1 


3.97 


8.4 


5.90 


Korea, Republic of 


551.7 


4.36 


528.3 


5.35 


23.4 


6.77 


Luxembourg 


501.9 


1.86 


484.8 


1.53 


17.2 


2.81 


Mexico 


390.8 


4.28 


380.0 


4.08 


10.9 


3.94 


Netherlands 


540.3 


4.08 


535.2 


3.48 


5.1 


4.29 


New Zealand 


530.7 


2.77 


516.2 


3.20 


14.5 


3.90 


Norway 


498.3 


2.85 


492.0 


2.87 


6.2 


3.21 


Poland 


493.0 


2.97 


487.5 


2.95 


5.6 


3.14 


Portugal 


472.4 


4.18 


460.2 


3.44 


12.2 


3.31 


Slovak Republic 


507.3 


3.90 


488.6 


3.59 


18.7 


3.65 


Spain 


489.6 


3.38 


480.7 


2.19 


8.9 


2.98 


Sweden 


512.3 


2.98 


505.8 


3.09 


6.5 


3.27 


Switzerland 


534.6 


4.74 


518.0 


3.65 


16.6 


4.87 


Turkey 


430.2 


7.88 


415.1 


6.68 


15.1 


6.16 


United States 


485.9 


3.33 


479.7 


3.24 


6.3 


2.89 


Non-OECD countries 


Hong Kong-China 


552.4 


6.52 


548.3 


4.56 


4.1 


6.65 


Indonesia 


361.8 


3.92 


358.5 


4.57 


3.3 


3.39 


Latvia 


484.8 


4.79 


482.0 


3.56 


2.8 


3.97 


Liechtenstein 


549.8 


7.23 


521.0 


6.28 


28.8 


10.92 


Macao-China 


538.2 


4.82 


516.9 


3.28 


21.3 


5.83 


Russian Federation 


473.5 


5.27 


463.4 


4.21 


10.1 


4.36 


Serbia and Montenegro 


437.5 


4.24 


436.3 


4.45 


1.2 


4.36 


Thailand 


414.7 


3.96 


418.8 


3.40 


-4.0 


4.24 


Tunisia 


364.9 


2.74 


352.7 


2.93 


12.2 


2.51 


Uruguay 


428.4 


3.95 


416.3 


3.84 


12.1 


4.15 


United Kingdom 1 


511.8 


2.90 


505.1 


3.88 


6.7 


4.90 



'Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE:The male-female score point difference is calculated by subtracting the average scores of females 
from the average scores of males. The OECD average is the average of the national averages of the OECD 
member countries with data available. Because PISA is principally an OECD study, the results for non- 
OECD countries are displayed separately from those of the OECD countries and are not included in the 
OECD average, s.e. means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



PISA 2003 Results From the U.S. Perspective 



Table B-19. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the combined mathematics literacy scale, by sex and country: 2003 







Below level 1 






Level 1 






Country 


Male 




Female 




Male 




Female 






Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


Percent 


s.e. 


OECD average 


8.1 


0.21 


8.4 


0.19 


12.6 


0.19 


13.8 


0.21 


OECD countries 


Australia 


4.6 


0.61 


4.0 


0.53 


10.3 


0.81 


9.7 


0.67 


Austria 


6.1 


0.95 


5.1 


0.72 


13.1 


0.99 


13.3 


1.20 


Belgium 


7.4 


0.82 


6.9 


0.82 


9.8 


0.87 


8.8 


0.77 


Canada 


2.9 


0.39 


2.0 


0.28 


7.4 


0.48 


7.4 


0.49 


Czech Republic 


4.3 


0.67 


5.7 


1.10 


10.9 


1.07 


12.3 


1.29 


Denmark 


3.8 


0.57 


5.6 


0.78 


9.6 


0.88 


11.8 


0.93 


Finland 


1.6 


0.33 


1.4 


0.29 


5.8 


0.59 


4.9 


0.56 


France 


6.1 


0.96 


5.2 


0.73 


10.7 


1.02 


11.3 


0.97 


Germany 


8.9 


1.04 


9.2 


1.05 


12.5 


0.99 


12.1 


1.02 


Greece 


16.4 


1.33 


19.1 


1.53 


19.4 


1.30 


22.8 


1.47 


Hungary 


7.6 


0.84 


8.0 


1.07 


14.6 


1.04 


15.9 


1.08 


Iceland 


6.1 


0.62 


2.8 


0.49 


12.1 


0.88 


8.8 


0.79 


Ireland 


4.2 


0.79 


5.2 


0.74 


10.8 


1.13 


13.5 


1.28 


Italy 


12.5 


1.55 


13.9 


1.70 


17.2 


1.57 


20.1 


1.33 


Japan 


5.2 


0.90 


4.3 


0.74 


9.1 


0.94 


8.1 


0.87 


Korea, Republic of 


2.3 


0.42 


2.7 


0.49 


6.2 


0.81 


8.3 


0.99 


Luxembourg 


6.8 


0.64 


8.0 


0.73 


13.2 


0.80 


15.3 


1.13 


Mexico 


36.2 


2.12 


39.7 


1.87 


26.9 


1.61 


28.8 


1.27 


Netherlands 


2.2 


0.70 


2.9 


0.84 


8.0 


1.23 


8.7 


1.18 


New Zealand 


4.7 


0.57 


5.2 


0.70 


9.9 


0.81 


10.4 


1.00 


Norway 


7.3 


0.71 


6.5 


0.76 


13.3 


0.90 


14.5 


1.07 


Poland 


7.7 


0.88 


5.9 


0.71 


14.9 


0.92 


15.5 


1.05 


Portugal 


12 


1.41 


10.6 


1.21 


16.7 


1.12 


20.6 


1.32 


Slovak Republic 


6.1 


0.91 


7.4 


0.94 


12.0 


1.07 


14.5 


1.25 


Spain 


8.4 


0.89 


7.8 


0.69 


14.1 


1.11 


15.7 


0.99 


Sweden 


5.6 


0.63 


5.6 


0.71 


11.1 


0.85 


12.3 


0.81 


Switzerland 


4.4 


0.46 


5.5 


0.65 


9.1 


0.76 


10.2 


0.83 


Turkey 


26.4 


2.35 


29.2 


2.37 


22.9 


1.51 


26.6 


1.82 


United States 


10.5 


0.96 


9.9 


1.03 


14.7 


0.82 


16.4 


1.16 


Non-OECD countries 


Hong Kong-China 


5.1 


1.12 


2.7 


0.71 


6.7 


0.87 


6.3 


0.87 


Indonesia 


49.2 


2.22 


51.8 


2.22 


28.8 


1.29 


26.5 


1.30 


Latvia 


8.1 


1.56 


7.2 


1.05 


16.3 


1.50 


15.9 


1.28 


Liechtenstein 


4.7 


1.79 


4.9 


2.37 


5.5 


2.16 


9.6 


3.03 


Macao-China 


2.3 


1.07 


2.4 


0.77 


8.5 


1.83 


9.1 


1.72 


Russian Federation 


11.4 


1.46 


11.4 


1.02 


18.4 


1.47 


19.2 


1.40 


Serbia and Montenegro 


19.2 


1.69 


16.1 


1.65 


24.1 


1.44 


24.8 


1.57 


Thailand 


25.3 


1.70 


22.6 


1.49 


29.7 


1.52 


30.6 


1.82 


Tunisia 


48.2 


1.71 


53.8 


1.68 


28.1 


1.38 


25.8 


1.39 


Uruguay 


24.7 


1.56 


27.7 


1.58 


20.9 


0.95 


22.7 


1.28 


United Kingdom 1 


5.3 


0.65 


5.2 


0.72 


12.1 


0.96 


12.9 


0.89 



See notes at end of table. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-19. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the combined mathematics literacy scale, by sex and country: 2003 — Continued 







Level 2 








Level 3 






Country 


Male 




Female 




Male 




Female 






Percent 


s . e . 


Percent 


s . e . 


Percent 


s . e . 


Percent 


s . e . 


OECD average 


20.0 


0.20 


22.1 


0.23 


22.9 


0.23 


24.5 


0.22 


OECD countries 


Australia 


17.8 


0.95 


19.4 


0.78 


22.7 


1.19 


25.3 


0.79 


Austria 


20.4 


1.11 


22.7 


1.35 


23.3 


1.47 


26.5 


1.39 


Belgium 


15.1 


0.94 


16.9 


0.88 


18.6 


0.82 


21.8 


1.00 


Canada 


16.1 


0.91 


18.7 


0.80 


23.4 


0.85 


28.6 


1.20 


Czech Republic 


19.8 


1.25 


20.4 


1.42 


23.2 


1.06 


25.4 


1.38 


Denmark 


18.7 


1.30 


22.3 


1.09 


26.4 


1.28 


26.0 


1.20 


Finland 


15.4 


0.75 


16.7 


0.80 


25.9 


0.93 


29.5 


1.09 


France 


18.7 


1.01 


21.6 


1.12 


25.1 


1.47 


26.6 


1.37 


Germany 


18.1 


1.21 


19.9 


1.42 


21.4 


0.98 


23.9 


1.37 


Greece 


24.7 


1.36 


27.8 


1.15 


21.0 


1.05 


19.4 


1.53 


Hungary 


23.6 


1.44 


24.0 


1.61 


23.9 


1.30 


24.7 


1.26 


Iceland 


20.4 


1.17 


20.1 


1.40 


25.3 


1.31 


26.9 


1.19 


Ireland 


22.5 


1.44 


24.7 


1.37 


27.8 


1.46 


28.2 


1.36 


Italy 


22.8 


1.27 


26.4 


1.35 


22.7 


1.12 


23.1 


1.17 


Japan 


15.8 


1.15 


16.9 


1.10 


20.2 


1.41 


24.5 


1.19 


Korea, Republic of 


14.6 


1.01 


19.6 


1.66 


22.3 


1.04 


26.7 


1.51 


Luxembourg 


21.4 


1.06 


24.4 


1.22 


24.8 


1.09 


26.9 


1.12 


Mexico 


21.6 


1.46 


20.2 


1.32 


11.4 


1.00 


8.9 


1.15 


Netherlands 


18.2 


1.49 


17.9 


1.42 


22.9 


1.57 


23.0 


1.34 


New Zealand 


17.7 


0.91 


20.6 


1.15 


21.9 


1.17 


24.5 


1.17 


Norway 


23.2 


1.25 


24.1 


1.54 


23.9 


1.35 


26.5 


1.20 


Poland 


22.9 


1.06 


26.8 


0.99 


24.5 


1.22 


26.2 


1.13 


Portugal 


24.6 


1.20 


29.4 


1.33 


23.9 


1.20 


24.1 


1.37 


Slovak Republic 


22.0 


1.10 


25.0 


1.42 


24.5 


1.28 


25.4 


1.46 


Spain 


23.3 


1.17 


26.1 


0.98 


25.6 


1.49 


27.7 


1.06 


Sweden 


21.3 


1.13 


22.1 


1.02 


25.4 


1.51 


25.6 


1.02 


Switzerland 


16.5 


1.15 


18.6 


1.11 


23.2 


1.48 


25.4 


1.17 


Turkey 


22.2 


1.28 


21.9 


1.75 


14.3 


1.45 


12.4 


1.47 


United States 


23.2 


1.05 


24.6 


1.37 


23.1 


1.39 


24.5 


1.11 


Non-OECD countries 


Hong Kong-China 


13.0 


0.95 


14.9 


1.57 


18.1 


1.02 


21.8 


2.29 


Indonesia 


15.2 


1.25 


14.3 


1.23 


5.1 


0.64 


5.8 


0.94 


Latvia 


24.6 


1.44 


26.3 


1.52 


25.6 


1.48 


26.9 


1.70 


Liechtenstein 


15.6 


3.07 


19.2 


3.92 


19.6 


3.48 


23.6 


3.71 


Macao-China 


16.8 


1.74 


22.3 


2.12 


23.7 


2.59 


29.8 


2.60 


Russian Federation 


24.5 


1.56 


28.3 


1.52 


22.6 


1.48 


23.6 


1.29 


Serbia and Montenegro 


26.3 


1.47 


30.9 


1.45 


17.5 


1.19 


20.2 


1.47 


Thailand 


24.5 


1.41 


26.1 


1.37 


13.2 


1.34 


14.0 


0.99 


Tunisia 


15.1 


1.00 


14.3 


1.00 


6.3 


0.66 


5.1 


0.87 


Uruguay 


24.3 


1.27 


24.1 


1.50 


17.4 


1.05 


16.1 


1.10 


United Kingdom 1 


19.5 


1.54 


22.7 


1.38 


25.8 


1.52 


25.4 


1.04 



See notes at end of table. 



PISA 2003 Results From the U.S. Perspective 



Table B-19. Percentage distribution of 15-year-old students scoring at each proficiency level on 
the combined mathematics literacy scale, by sex and country: 2003 — Continued 







Level 4 








Level 5 






Country 


Male 




Female 




Male 




Female 






Percent 


s . e . 


Percent 


s . e . 


Percent 


s . e . 


Percent 


s . e . 


OECD average 


19.5 


0.22 


18.8 


0.24 


11.8 


0.19 


9.5 


0.15 


OECD countries 


Australia 


22.9 


1.07 


23.6 


1.03 


14.6 


0.96 


13.4 


0.81 


Austria 


20.4 


1.28 


20.5 


1.43 


11.9 


1.04 


9.2 


1.04 


Belgium 


20.1 


0.86 


22.1 


0.84 


18.1 


1.04 


16.7 


0.75 


Canada 


25.0 


0.70 


25.4 


1.04 


17.6 


0.88 


13.6 


0.82 


Czech Republic 


20.3 


0.99 


21.3 


1.33 


15.0 


1.12 


10.8 


0.98 


Denmark 


23.5 


1.11 


20.4 


1.26 


13.1 


1.02 


10.6 


1.05 


Finland 


25.4 


1.14 


26.9 


1.22 


17.7 


1.08 


15.7 


0.83 


France 


21.6 


1.51 


22.6 


1.06 


13.3 


1.16 


10.1 


1.02 


Germany 


20.7 


1.32 


20.6 


1.18 


13.0 


1.14 


11.3 


0.97 


Greece 


12.8 


1.24 


8.6 


0.83 


4.8 


0.80 


2.1 


0.46 


Hungary 


18.3 


1.20 


18.1 


1.11 


8.6 


0.75 


7.7 


1.01 


Iceland 


21.0 


1.17 


25.5 


1.10 


11.4 


0.92 


12.2 


0.95 


Ireland 


21.0 


1.63 


19.4 


1.21 


10.8 


1.09 


7.4 


0.83 


Italy 


15.1 


1.06 


11.9 


0.77 


7.1 


0.64 


3.9 


0.39 


Japan 


22.3 


1.42 


24.9 


1.62 


16.5 


1.37 


15.6 


1.19 


Korea, Republic of 


25.9 


1.42 


23.6 


1.53 


18.9 


1.20 


13.4 


1.21 


Luxembourg 


20.0 


1.11 


17.4 


1.08 


10.5 


0.93 


6.6 


0.67 


Mexico 


3.4 


0.53 


2.1 


0.52 


0.5 


0.15 


0.2 


0.07 


Netherlands 


22.6 


1.65 


22.5 


1.51 


18.1 


1.51 


18.3 


1.21 


New Zealand 


21.9 


1.18 


21.8 


1.08 


15.7 


1.00 


12.4 


0.99 


Norway 


19.1 


1.23 


18.7 


1.17 


9.7 


0.79 


7.7 


0.69 


Poland 


17.9 


1.23 


17.5 


1.06 


9.0 


0.89 


6.5 


0.78 


Portugal 


15.6 


1.56 


11.5 


1.01 


5.9 


0.82 


3.3 


0.59 


Slovak Republic 


20.0 


1.20 


17.8 


1.00 


11.4 


0.93 


8.1 


0.76 


Spain 


18.7 


1.08 


16.7 


0.91 


8.0 


1.09 


5.1 


0.48 


Sweden 


19.4 


1.43 


20.2 


1.19 


12.4 


1.01 


10.9 


1.03 


Switzerland 


22.6 


1.25 


22.3 


1.16 


15.2 


1.80 


13.1 


1.15 


Turkey 


7.5 


1.14 


5.8 


1.25 


3.5 


0.91 


2.6 


0.83 


United States 


16.9 


1.08 


16.2 


1.01 


8.9 


0.70 


7.2 


0.79 


Non-OECD countries 


Hong Kong-China 


23.9 


1.58 


26.1 


1.29 


20.4 


1.52 


19.9 


1.56 


Indonesia 


1.4 


0.36 


1.3 


0.49 


0.2 


0.10 


0.2 


0.13 


Latvia 


16.1 


1.54 


17.0 


1.28 


7.1 


1.01 


5.6 


0.79 


Liechtenstein 


22.2 


4.93 


24.2 


4.45 


21.5 


5.54 


15.0 


3.60 


Macao-China 


24.7 


3.18 


22.7 


2.17 


17.2 


3.28 


10.7 


1.45 


Russian Federation 


14.1 


1.11 


12.3 


1.15 


6.6 


0.86 


4.2 


0.59 


Serbia and Montenegro 


9.6 


0.97 


6.7 


1.16 


2.9 


0.62 


1.3 


0.35 


Thailand 


5.6 


0.77 


5.1 


0.72 


1.5 


0.41 


1.4 


0.45 


Tunisia 


2.0 


0.42 


0.9 


0.36 


0.3 


0.19 


0.2 


0.10 


Uruguay 


8.9 


0.76 


7.4 


0.88 


3.0 


0.45 


1.7 


0.35 


United Kingdom 1 


21.4 


0.87 


19.9 


1.14 


11.6 


1.15 


10.5 


1.14 



See notes at end of table. 



International Outcomes of Learning in Mathematics Literacy and Problem Solving 



Table B-19. Percentage distribution of 15-year-old students scoring at each proficiency level 
on the combined mathematics literacy scale, by sex and country: 2003 — Continued 






Level 6 






Country 


Male 




Female 






Percent 


s.e. 


Percent 


s.e. 


OECD average 


5.1 


0.14 


2.9 


0.10 


OECD countries 


Australia 


7.0 


0.74 


4.5 


0.49 


Austria 


4.8 


0.75 


2.7 


0.51 


Belgium 


10.9 


0.72 


6.8 


0.54 


Canada 


7.5 


0.79 


4.2 


0.36 


Czech Republic 


6.6 


0.72 


4.1 


0.54 


Denmark 


4.9 


0.62 


3.3 


0.56 


Finland 


8.2 


0.85 


5.1 


0.53 


France 


4.6 


0.58 


2.5 


0.56 


Germany 


5.3 


0.65 


2.9 


0.57 


Greece 


1.0 


0.33 


0.2 


0.14 


Hungary 


3.3 


0.56 


1.6 


0.37 


Iceland 


3.7 


0.53 


3.8 


0.49 


Ireland 


2.9 


0.50 


1.6 


0.36 


Italy 


2.5 


0.30 


0.7 


0.13 


Japan 


10.9 


1.94 


5.7 


0.78 


Korea, Republic of 


9.7 


1.01 


5.7 


1.15 


Luxembourg 


3.4 


0.56 


1.4 


0.32 


Mexico 


# 


t 


# 


t 


Netherlands 


8.0 


0.82 


6.6 


0.70 


New Zealand 


8.3 


0.73 


5.0 


0.58 


Norway 


3.5 


0.52 


1.9 


0.43 


Poland 


3.1 


0.50 


1.5 


0.30 


Portugal 


1.3 


0.26 


0.4 


0.18 


Slovak Republic 


4.1 


0.58 


1.7 


0.30 


Spain 


1.9 


0.39 


1.0 


0.29 


Sweden 


4.9 


0.70 


3.4 


0.58 


Switzerland 


9.0 


1.30 


4.9 


0.89 


Turkey 


3.0 


1.24 


1.6 


0.86 


United States 


2.8 


0.52 


1.2 


0.39 


Non-OECD countries 


Hong Kong-China 


12.7 


1.46 


8.3 


0.96 


Indonesia 


# 


t 


# 


t 


Latvia 


2.3 


0.53 


1.1 


0.33 


Liechtenstein 


10.8 


2.66 


3.6 


1.78 


Macao-China 


6.8 


1.86 


3.0 


0.91 


Russian Federation 


2.3 


0.63 


1.0 


0.31 


Serbia and Montenegro 


0.4 


0.21 


0.1 


0.09 


Thailand 


0.2 


0.12 


0.2 


0.11 


Tunisia 


# 


t 


# 


t 


Uruguay 


0.8 


0.26 


0.2 


0.13 


United Kingdom 1 


4.4 


0.59 


3.4 


0.68 



f Not applicable. 

# Rounds to zero. 

'Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE: In order to reach a particular level, a student must have been able to correctly answer a majority of items at that level. 
Students were classified into mathematics literacy levels according to their scores. Exact cut point scores are as follows: 
below level 1 (a score less than or equal to 357.7)\ level 1 (a score greater than 357.7 and less than or equal to 420.07 ); level 2 (a 
score greated than 420.07 and less than or equal to 482,38)', level 3 (a score greater than 482.38 and less than or equal to 544.68)', 
level 4 (a score greater than 544.68 and less than or equal to 606.99)', level 5 (a score greater than 606.99 and less than or equal to 
669.3)', level 6 (a score greater than 669.3). The OECD average is the average of the national averages of the OECD member 
countries with data available. Because PISA is principally an OECD study, the results for non-OECD countries are displayed 
separately from those of the OECD countries and are not included in the OECD average, s.e. means standard error. Detail may 
not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 



PISA 2003 Results From the U.S. Perspective 



Table B-20. Average mathematics literacy subscale scores of 15-year-old 
students, by sex and country: 2003 









Space and shape 






Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


504.6 


0.81 


487.9 


0.79 


16.7 


0.90 


OECD countries 


Australia 


526.4 


3.15 


514.5 


2.91 


11.9 


3.88 


Austria 


524.5 


4.41 


505.8 


4.34 


18.7 


5.18 


Belgium 


538.1 


3.23 


520.3 


3.25 


17.9 


4.62 


Canada 


530.1 


2.07 


510.6 


2.18 


19.5 


2.52 


Czech Republic 


542.3 


4.76 


512.1 


5.05 


30.2 


5.68 


Denmark 


520.7 


3.41 


504.4 


3.26 


16.3 


3.73 


Finland 


540.3 


2.63 


537.8 


2.40 


2.4 


2.96 


France 


517.0 


4.26 


499.2 


3.25 


17.8 


4.72 


Germany 


505.8 


4.03 


494.3 


3.99 


11.5 


4.69 


Greece 


447.1 


4.75 


427.8 


3.82 


19.3 


4.02 


Hungary 


486.1 


3.78 


471.1 


3.92 


15.0 


4.04 


Iceland 


496.2 


2.43 


511.3 


2.30 


-15.1 


3.74 


Ireland 


488.9 


2.96 


463.4 


3.44 


25.5 


4.28 


Italy 


479.7 


4.74 


461.5 


4.06 


18.1 


6.33 


Japan 


557.8 


6.30 


548.9 


4.20 


8.9 


6.35 


Korea, Republic of 


562.7 


5.14 


535.7 


6.24 


27.0 


7.96 


Luxembourg 


502.5 


2.21 


474.3 


2.02 


28.3 


3.28 


Mexico 


389.8 


4.06 


374.2 


3.48 


15.6 


3.84 


Netherlands 


530.1 


3.70 


522.0 


3.45 


8.2 


4.28 


New Zealand 


533.9 


2.70 


516.0 


3.35 


17.9 


3.89 


Norway 


486.3 


3.10 


479.0 


3.53 


7.3 


4.27 


Poland 


496.9 


3.20 


483.7 


3.29 


13.1 


3.70 


Portugal 


458.1 


4.25 


443.0 


3.53 


15.1 


3.52 


Slovak Republic 


522.4 


4.75 


487.4 


4.11 


35.0 


4.48 


Spain 


485.9 


3.52 


467.4 


2.35 


18.5 


2.96 


Sweden 


503.5 


3.01 


493.1 


3.21 


10.4 


3.52 


Switzerland 


551.7 


5.30 


526.5 


3.70 


25.3 


5.57 


Turkey 


422.7 


7.59 


410.9 


6.15 


11.7 


5.99 


United States 


479.5 


3.34 


464.3 


3.09 


15.2 


3.24 


Non-OECD countries 


Hong Kong-China 


560.5 


6.79 


556.4 


4.95 


4.1 


6.85 


Indonesia 


368.7 


3.69 


353.0 


4.21 


15.7 


2.86 


Latvia 


493.7 


5.22 


479.7 


3.91 


14.0 


4.19 


Liechtenstein 


557.0 


7.94 


518.4 


7.08 


38.5 


12.13 


Macao-China 


539.8 


5.13 


516.6 


4.30 


23.3 


6.76 


Russian Federation 


484.7 


5.78 


464.1 


4.97 


20.6 


5.03 


Serbia and Montenegro 


434.1 


4.28 


430.8 


4.93 


3.3 


4.88 


Thailand 


426.4 


4.35 


421.8 


3.83 


4.5 


4.67 


Tunisia 


367.2 


2.81 


350.9 


3.15 


16.3 


3.02 


Uruguay 


422.8 


3.59 


401.7 


3.37 


21.1 


3.61 


United Kingdom 1 


501.5 


3.03 


491.2 


3.97 


10.3 


5.04 



See notes at end of table. 
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Table B-20. Average mathematics literacy subscale scores of 15-year-old 
students, by sex and country: 2003 — Continued 









Change and relationships 






Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


504.3 


0.85 


493.3 


0.83 


11.0 


0.89 


OECD countries 


Australia 


527.5 


3.18 


523.1 


2.78 


4.4 


3.81 


Austria 


502.1 


4.36 


497.4 


4.42 


4.6 


4.99 


Belgium 


538.9 


3.55 


531.3 


3.50 


7.6 


5.08 


Canada 


545.8 


2.20 


532.4 


2.01 


13.5 


2.29 


Czech Republic 


521.2 


4.48 


508.4 


3.98 


12.8 


4.93 


Denmark 


519.9 


3.67 


499.1 


3.25 


20.8 


3.50 


Finland 


548.8 


2.80 


537.4 


2.39 


11.4 


2.82 


France 


522.0 


4.02 


517.6 


3.18 


4.4 


4.97 


Germany 


513.5 


4.28 


501.8 


4.37 


11.8 


4.43 


Greece 


444.8 


5.18 


427.0 


4.42 


17.8 


4.18 


Hungary 


499.2 


3.63 


489.5 


3.62 


9.7 


3.88 


Iceland 


504.8 


2.43 


514.4 


2.30 


-9.6 


3.79 


Ireland 


512.2 


3.05 


499.6 


3.52 


12.6 


4.44 


Italy 


462.9 


4.85 


442.1 


3.99 


20.8 


6.27 


Japan 


539.4 


6.41 


533.1 


4.26 


6.3 


6.56 


Korea, Republic of 


557.8 


4.68 


532.5 


5.80 


25.3 


7.29 


Luxembourg 


494.0 


2.51 


480.2 


1.76 


13.8 


3.67 


Mexico 


368.2 


4.87 


360.3 


4.56 


7.9 


4.43 


Netherlands 


554.3 


3.84 


548.4 


3.71 


5.9 


4.27 


New Zealand 


534.4 


2.79 


517.0 


3.43 


17.4 


4.06 


Norway 


489.8 


3.18 


485.5 


3.05 


4.3 


3.33 


Poland 


488.2 


3.15 


480.5 


3.36 


7.7 


3.63 


Portugal 


474.7 


4.82 


461.6 


4,00 


13.1 


3.80 


Slovak Republic 


502.4 


4.09 


486.0 


3.90 


16.4 


4.16 


Spain 


485.0 


3.81 


476.6 


2.58 


8.4 


3.25 


Sweden 


505.8 


3.42 


504.4 


3.88 


1.4 


4.33 


Switzerland 


529.9 


5.11 


515.0 


3.95 


14.9 


5.30 


Turkey 


425.5 


9.07 


419.5 


7.43 


6.0 


7.25 


United States 


488.3 


3.42 


482.7 


3.29 


5.6 


2.90 


Non-OECD countries 


Hong Kong-China 


540.2 


6.83 


539.2 


4.77 


1.0 


7.16 


Indonesia 


336.0 


4.35 


331.7 


5.38 


4.3 


3.45 


Latvia 


486.7 


5.27 


487.7 


4.31 


-1.0 


3.98 


Liechtenstein 


552.0 


7.45 


526.4 


6.49 


25.6 


12.10 


Macao-China 


529.1 


5.03 


509.0 


4.59 


20.1 


6.56 


Russian Federation 


478.5 


6.02 


475.0 


4.47 


3.4 


5.05 


Serbia and Montenegro 


419.8 


4.47 


418.3 


4.87 


1.4 


4.93 


Thailand 


399.7 


4.52 


409.3 


3.95 


-9.6 


5.06 


Tunisia 


342.3 


2.99 


331.0 


3.35 


11.3 


3.01 


Uruguay 


419.7 


4.24 


414.4 


4.21 


5.2 


4.38 


United Kingdom 1 


517.3 


2.97 


509.0 


4.04 


8.3 


5.04 
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Table B-20. Average mathematics literacy subscale scores of 15-year-old 
students, by sex and country: 2003 — Continued 









Quantity 








Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


503.8 


0.77 


497.6 


0.77 


6.2 


0.84 


OECD countries 


Australia 


517.5 


2.85 


516.3 


2.65 


1.2 


3.67 


Austria 


514.7 


3.68 


511.6 


3.66 


3.1 


4.19 


Belgium 


530.1 


3.30 


529.1 


3.26 


0.9 


4.66 


Canada 


532.7 


2.16 


528.0 


1.93 


4.7 


2.23 


Czech Republic 


530.8 


4.16 


525.1 


4.55 


5.8 


5.11 


Denmark 


520.3 


3.21 


511.0 


2.89 


9.3 


3.07 


Finland 


550.1 


2.26 


546.9 


2.07 


3.2 


2.33 


France 


508.1 


3.78 


505.8 


2.88 


2.3 


4.45 


Germany 


514.5 


4.19 


513.9 


3.85 


0.6 


4.40 


Greece 


457.6 


4.91 


435.0 


3.97 


22.6 


3.99 


Hungary 


497.2 


3.27 


495.3 


3.24 


1.9 


3.61 


Iceland 


499.6 


2.53 


528.0 


2.32 


-28.5 


3.89 


Ireland 


506.1 


3.06 


497.2 


3.47 


8.9 


4.28 


Italy 


481.4 


4.96 


468.7 


4.38 


12.7 


6.54 


Japan 


528.3 


5.59 


525.2 


3.73 


3.1 


5.67 


Korea, Republic of 


546.1 


3.96 


524.2 


4.91 


21.9 


6.15 


Luxembourg 


505.8 


2.17 


497.3 


1.62 


8.5 


3.17 


Mexico 


400.0 


4.81 


388.0 


4.31 


12.0 


4.46 


Netherlands 


526.3 


4.18 


530.3 


3.57 


-4.0 


4.73 


New Zealand 


516.9 


2.66 


505.3 


3.24 


11.6 


3.89 


Norway 


494.2 


2.85 


494.2 


2.70 


# 


t 


Poland 


492.6 


2.94 


491.0 


2.98 


1.6 


3.27 


Portugal 


472.7 


4.12 


458.9 


3.69 


13.8 


3.27 


Slovak Republic 


518.6 


4.03 


506.1 


3.63 


12.6 


3.64 


Spain 


494.8 


3.60 


490.0 


2.18 


4.8 


3.07 


Sweden 


515.2 


2.92 


511.9 


3.24 


3.2 


3.63 


Switzerland 


536.0 


4.41 


529.0 


3.23 


7.0 


4.60 


Turkey 


421.1 


8.04 


403.5 


6.62 


17.5 


6.33 


United States 


478.5 


3.63 


474.3 


3.57 


4.2 


3.38 


Non-OECD countries 


Hong Kong-China 


543.9 


6.05 


546.5 


4.12 


-2.6 


6.09 


Indonesia 


358.5 


4.05 


356.5 


5.01 


2.1 


3.12 


Latvia 


483.2 


4.42 


480.3 


3.55 


2.9 


3.43 


Liechtenstein 


543.9 


7.00 


522.6 


5.65 


21.4 


9.89 


Macao-China 


541.6 


4.28 


524.9 


4.18 


16.7 


6.00 


Russian Federation 


475.7 


5.04 


469.3 


4.18 


6.4 


4.43 


Serbia and Montenegro 


454.8 


4.22 


457.9 


4.67 


-3.1 


4.71 


Thailand 


412.3 


4.15 


416.9 


3.84 


-4.5 


4.91 


Tunisia 


372.3 


2.90 


356.6 


3.30 


15.6 


2.69 


Uruguay 


435.9 


3.91 


423.8 


3.76 


12.0 


4.09 


United Kingdom 1 


499.6 


2.87 


497.6 


4.03 


2.1 


4.95 




See notes at end of table. 
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Table B-20. Average mathematics literacy subscale scores of 15-year-old students, 
by sex and country: 2003 — Continued 









Uncertainty 








Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


508.3 


0.71 


495.6 


0.75 


12.6 


0.78 


OECD countries 


Australia 


534.5 


3.03 


527.2 


2.74 


7.3 


3.72 


Austria 


497.7 


3.81 


489.9 


3.99 


7.8 


4.58 


Belgium 


529.2 


3.23 


521.8 


3.21 


7.3 


4.68 


Canada 


551.4 


2.20 


538.4 


1.89 


13.0 


2.26 


Czech Republic 


508.5 


3.88 


491.8 


3.77 


16.7 


4.63 


Denmark 


526.6 


3.40 


505.0 


3.03 


21.6 


3.21 


Finland 


550.8 


2.64 


538.7 


2.27 


12.1 


2.63 


France 


511.7 


3.53 


501.0 


2.83 


10.7 


4.25 


Germany 


502.0 


3.86 


483.9 


3.79 


18.1 


3.97 


Greece 


468.9 


4.29 


448.7 


3.68 


20.2 


3.68 


Hungary 


492.8 


3.16 


484.8 


3.00 


7.9 


3.34 


Iceland 


524.2 


2.45 


531.7 


2.35 


-7.5 


3.76 


Ireland 


524.9 


3.24 


509.4 


3.73 


15.5 


4.60 


Italy 


475.1 


4.51 


451.1 


3.80 


24.1 


5.94 


Japan 


535.2 


5.57 


521.1 


3.82 


14.0 


5.69 


Korea, Republic of 


547.1 


4.12 


525.4 


5.18 


21.7 


6.62 


Luxembourg 


503.1 


2.23 


481.5 


1.82 


21.7 


3.49 


Mexico 


392.1 


3.79 


387.7 


3.63 


4.5 


3.51 


Netherlands 


554.0 


3.56 


544.5 


3.69 


9.5 


4.10 


New Zealand 


538.0 


2.73 


526.5 


3.31 


11.5 


3.93 


Norway 


517.9 


2.97 


507.6 


3.16 


10.3 


3.32 


Poland 


494.8 


2.82 


492.3 


2.84 


2.6 


3.15 


Portugal 


475.6 


4.10 


466.0 


3.45 


9.6 


3.15 


Slovak Republic 


484.1 


3.80 


467.1 


3.41 


17.0 


3.49 


Spain 


493.0 


3.34 


485.0 


2.17 


8.0 


2.83 


Sweden 


515.2 


3.18 


506.4 


3.38 


8.8 


3.66 


Switzerland 


526.4 


4.73 


505.9 


3.68 


20.5 


5.16 


Turkey 


451.1 


7.31 


432.2 


6.07 


19.0 


5.74 


United States 


493.0 


3.41 


489.8 


3.15 


3.2 


2.79 


Non-OECD countries 


Hong Kong-China 


564.1 


6.57 


552.4 


4.58 


11.8 


6.66 


Indonesia 


382.1 


2.81 


386.9 


3.38 


-4.8 


2.42 


Latvia 


473.7 


4.23 


473.9 


3.08 


-0.2 


3.32 


Liechtenstein 


538.4 


6.94 


507.6 


5.62 


30.8 


10.46 


Macao-China 


540.7 


4.51 


523.0 


4.20 


17.8 


5.88 


Russian Federation 


440.7 


5.13 


432.3 


3.94 


8.4 


4.19 


Serbia and Montenegro 


430.6 


4.01 


425.2 


4.19 


5.4 


4.25 


Thailand 


419.9 


3.44 


424.9 


3.01 


-5.0 


4.00 


Tunisia 


366.7 


2.50 


360.1 


2.77 


6.7 


2.58 


Uruguay 


422.9 


3.85 


414.5 


3.60 


8.3 


4.09 


United Kingdom 1 


523.1 


2.90 


517.5 


3.84 


5.6 


4.87 



t Not applicable. 

# Rounds to zero. 

’Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE:The male-female score point difference is calculated by subtracting the average scores of females from 
the average scores of males. The OECD average is the average of the national averages of the OECD member 
countries with data available. Because PISA is principally an OECD study, the results for non-OECD countries 
are displayed separately from those of the OECD countries and are not included in the OECD average, s.e. 
means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2003. 
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Table B-21. Average problem-solving scores of 15-year-old students, by sex 
and country: 2003 



Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


499.2 


0.76 


500.9 


0.77 


-1.7 


0.82 


OECD countries 


Australia 


526.7 


2.68 


533.1 


2.48 


-6.4 


3.33 


Austria 


504.7 


3.92 


507.6 


3.75 


-2.9 


4.34 


Belgium 


523.6 


3.14 


527.1 


3.15 


-3.5 


4.52 


Canada 


532.7 


2.01 


532.2 


1.82 


0.5 


2.06 


Czech Republic 


519.6 


4.10 


513.1 


4.34 


6.5 


4.99 


Denmark 


519.3 


3.08 


514.4 


2.93 


4.9 


3.20 


Finland 


542.6 


2.54 


552.6 


2.25 


-10.0 


3.03 


France 


518.7 


3.81 


519.5 


2.93 


-0.8 


4.14 


Germany 


511.0 


3.87 


516.7 


3.67 


-5.7 


3.90 


Greece 


449.5 


4.94 


447.5 


4.11 


1.9 


4.37 


Hungary 


499.3 


3.43 


503.0 


3.39 


-3.7 


3.71 


Iceland 


490.0 


2.25 


520.4 


2.49 


-30.5 


3.90 


Ireland 


498.7 


2.78 


498.2 


3.46 


0.5 


4.20 


Italy 


467.4 


5.04 


471.4 


3.54 


-4.1 


6.01 


Japan 


546.1 


5.74 


548.5 


4.07 


-2.4 


5.66 


Korea, Republic of 


553.7 


3.99 


545.6 


4.82 


8.1 


6.10 


Luxembourg 


494.9 


2.41 


492.5 


1.85 


2.4 


3.32 


Mexico 


387.0 


5.04 


381.9 


4.67 


5.1 


4.45 


Netherlands 


522.4 


3.60 


517.9 


3.58 


4.5 


4.09 


New Zealand 


531.1 


2.65 


534.4 


3.11 


-3.3 


3.82 


Norway 


485.6 


3.08 


494.1 


3.24 


-8.5 


3.57 


Poland 


486.1 


3.37 


487.1 


3.00 


-1.1 


3.13 


Portugal 


469.8 


4.58 


469.8 


3.90 


# 


t 


Slovak Republic 


495.1 


4.07 


488.2 


3.56 


6.9 


3.66 


Spain 


479.2 


3.64 


485.2 


2.59 


-6.0 


3.13 


Sweden 


503.6 


2.96 


513.5 


2.83 


-9.9 


3.12 


Switzerland 


520.1 


3.98 


522.6 


3.29 


-2.5 


4.11 


Turkey 


408.4 


7.31 


406.4 


5.84 


2.0 


5.83 


United States 


476.9 


3.44 


477.8 


3.50 


-0.9 


3.03 


Non-OECD countries 


Hong Kong-China 


545.4 


6.16 


550.4 


4.05 


-5.1 


6.27 


Indonesia 


357.7 


3.15 


365.0 


4.03 


-7.3 


3.01 


Latvia 


481.2 


5.07 


483.7 


3.98 


-2.6 


4.65 


Liechtenstein 


535.1 


6.65 


523.5 


5.92 


11.5 


9.84 


Macao-China 


538.2 


4.25 


526.9 


3.20 


11.2 


5.55 


Russian Federation 


479.7 


5.92 


477.4 


4.37 


2.3 


4.87 


Serbia and Montenegro 


416.5 


3.81 


423.9 


3.92 


-7.4 


4.07 


Thailand 


418.2 


3.87 


430.5 


3.07 


-12.4 


4.33 


Tunisia 


346.1 


2.50 


343.4 


2.45 


2.7 


2.57 


Uruguay 


412.1 


4.59 


409.3 


4.21 


2.7 


4.77 


United Kingdom 1 


505.7 


2.97 


514.1 


3.50 


-8.4 


4.51 



f Not applicable. 

# Rounds to zero. 

'Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE:The male-female score point difference is calculated by subtracting the average scores of females 
from the average scores of males. The OECD average is the average of the national averages of the OECD 
member countries with data available. Because PISA is principally an OECD study, the results for non- 
OECD countries are displayed separately from those of the OECD countries and are not included in the 
OECD average, s.e. means standard error. Detail may not sum to totals because of rounding. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 
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Table B-22. Average combined reading literacy and science literacy scores of 
15-year-old students, by sex and country: 2000 and 2003 









Reading literacy in 2000 






Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


484.8 


0.82 


516.5 


0.75 


-31.7 


0.94 


OECD countries 


Australia 


512.7 


4.04 


546.3 


4.74 


-33.6 


5.44 


Austria 


494.7 


3.23 


520.3 


3.59 


-25.6 


5.24 


Belgium 


492.4 


4.24 


525.2 


4.92 


-32.8 


5.99 


Canada 


518.9 


1.76 


551.1 


1.70 


-32.2 


1.63 


Czech Republic 


472.6 


4.11 


510.1 


2.53 


-37.4 


4.71 


Denmark 


485.4 


2.95 


510.3 


2.87 


-24.8 


3.28 


Finland 


520.1 


3.00 


571.4 


2.78 


-51.3 


2.63 


France 


490.3 


3.50 


519.1 


2.72 


-28.8 


3.38 


Germany 


467.6 


3.17 


502.2 


3.88 


-34.7 


5.21 


Greece 


455.7 


6.07 


492.7 


4.63 


-37.0 


5.01 


Hungary 


464.5 


5.34 


496.2 


4.35 


-31.6 


5.73 


Iceland 


488.5 


2.12 


528.1 


2.14 


-39.7 


3.11 


Ireland 


512.8 


4.18 


541.5 


3.55 


-28.7 


4.56 


Italy 


469.2 


5.14 


507.4 


3.57 


-38.2 


7.05 


Japan 


507.3 


6.74 


536.9 


5.39 


-29.7 


6.44 


Korea, Republic of 


518.5 


3.77 


532.7 


3.70 


-14.2 


6.02 


Luxembourg 


428.8 


2.58 


455.7 


2.30 


-26.9 


3.77 


Mexico 


411.5 


4.18 


431.8 


3.84 


-20.3 


4.34 


Netherlands 


— 


— 


— 


— 


— 


— 


New Zealand 


506.8 


4.18 


552.6 


3.80 


-45.8 


6.28 


Norway 


485.6 


3.79 


528.8 


2.86 


-43.2 


4.04 


Poland 


461.4 


5.99 


497.5 


5.52 


-36.1 


6.97 


Portugal 


457.7 


4.98 


482.4 


4.64 


-24.7 


3.77 


Slovak Republic 


— 


— 


— 


— 


— 


— 


Spain 


481.2 


3.35 


505.4 


2.76 


-24.1 


3.17 


Sweden 


498.6 


2.56 


535.6 


2.48 


-37.0 


2.70 


Switzerland 


480.1 


4.85 


510.0 


4.50 


-30.0 


4.17 


Turkey 


— 


— 


— 


— 


— 


— 


United States 


489.7 


8.41 


518.2 


6.20 


-28.6 


4.12 


Non-OECD countries 


Hong Kong-China 


— 


— 


— 


— 


— 


— 


Indonesia 


— 


— 


— 


— 


— 


— 


Latvia 


431.9 


5.53 


484.7 


5.40 


-52.8 


4.20 


Liechtenstein 


468.5 


7.33 


499.6 


6.83 


-31.2 


11.54 


Macao-China 


— 


— 


— 


— 


— 


— 


Russian Federation 


442.8 


4.53 


481.0 


4.09 


-38.2 


2.92 


Serbia and Montenegro 


— 


— 


— 


— 


— 


— 


Thailand 


— 


— 


— 


— 


— 


— 


Tunisia 


— 


— 


— 


— 


— 


— 


Uruguay 


— 


— 


— 


— 


— 


— 


United Kingdom 1 


511.6 


3.03 


537.2 


3.40 


-25.6 


4.28 



See notes at end of table. 
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Table B-22. Average combined reading literacy and science literacy scores of 
15-year-old students, by sex and country: 2000 and 2003 — Continued 









Reading literacy in 2003 






Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


477.3 


0.74 


511.4 


0.72 


-34.1 


0.78 


OECD countries 


Australia 


506.1 


2.84 


545.4 


2.55 


-39.3 


3.60 


Austria 


467.1 


4.54 


514.4 


4.21 


-47.2 


5.23 


Belgium 


489.3 


3.76 


526.2 


3.30 


-36.9 


5.10 


Canada 


514.0 


2.05 


545.5 


1.82 


-31.5 


2.02 


Czech Republic 


473.1 


4.11 


504.4 


4.42 


-31.3 


4.94 


Denmark 


479.4 


3.31 


504.8 


2.98 


-25.4 


2.88 


Finland 


521.4 


2.21 


565.4 


1.97 


-44.0 


2.66 


France 


476.1 


3.79 


514.3 


3.18 


-38.2 


4.47 


Germany 


470.8 


4.23 


512.9 


3.91 


-42.1 


4.62 


Greece 


452.9 


5.10 


490.4 


3.96 


-37.5 


4.12 


Hungary 


467.2 


3.16 


498.2 


3.05 


-31.0 


3.76 


Iceland 


463.8 


2.33 


521.6 


2.22 


-57.8 


3.47 


Ireland 


501.1 


3.26 


530.1 


3.71 


-29.0 


4.56 


Italy 


455.2 


5.06 


494.6 


3.40 


-39.3 


6.03 


Japan 


486.6 


5.48 


509.0 


4.07 


-22.4 


5.40 


Korea, Republic of 


525.5 


3.69 


546.7 


4.27 


-21.3 


5.59 


Luxembourg 


462.7 


2.60 


495.7 


1.84 


-33.0 


3.36 


Mexico 


388.6 


4.56 


410.1 


4.57 


-21.5 


4.36 


Netherlands 


502.9 


3.66 


523.8 


3.20 


-20.9 


3.93 


New Zealand 


507.7 


3.13 


535.4 


3.29 


-27.6 


4.37 


Norway 


475.3 


3.36 


524.5 


3.38 


-49.2 


3.73 


Poland 


476.8 


3.55 


516.3 


3.19 


-39.6 


3.66 


Portugal 


458.5 


4.28 


494.9 


3.72 


-36.3 


3.34 


Slovak Republic 


453.3 


3.82 


485.8 


3.32 


-32.5 


3.50 


Spain 


460.7 


3.78 


499.8 


2.48 


-39.1 


3.88 


Sweden 


495.9 


2.81 


532.7 


2.89 


-36.7 


3.15 


Switzerland 


482.0 


4.37 


517.5 


3.10 


-35.5 


4.69 


Turkey 


426.0 


6.80 


459.3 


6.12 


-33.3 


5.83 


United States 


479.3 


3.66 


511.3 


3.55 


-32.0 


3.35 


Non-OECD countries 


Hong Kong-China 


493.8 


5.31 


525.4 


3.49 


-31.5 


5.51 


Indonesia 


369.5 


3.35 


393.5 


3.92 


-24.0 


2.78 


Latvia 


470.4 


4.51 


509.1 


3.66 


-38.7 


4.25 


Liechtenstein 


516.6 


7.24 


534.0 


6.54 


-17.4 


11.87 


Macao-China 


490.8 


3.62 


504.1 


2.75 


-13.3 


4.77 


Russian Federation 


427.8 


4.73 


456.4 


3.69 


-28.5 


3.88 


Serbia and Montenegro 


389.9 


3.66 


433.0 


3.91 


-43.1 


3.93 


Thailand 


396.4 


3.72 


439.2 


3.01 


-42.7 


4.05 


Tunisia 


361.8 


3.31 


387.1 


3.27 


-25.3 


3.56 


Uruguay 


414.0 


4.50 


453.3 


3.72 


-39.3 


4.71 


United Kingdom 1 


491.8 


3.07 


520.4 


3.62 


-28.6 


4.81 



See notes at end of table. 
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Table B-22. Average combined reading literacy and science literacy scores of 
15-year-old students, by sex and country: 2000 and 2003 — Continued 









Science literacy in 2000 






Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


500.5 


0.88 


500.7 


0.76 


0.2 


1.01 


OECD countries 


Australia 


526.4 


3.91 


528.9 


4.78 


-2.5 


5.29 


Austria 


525.7 


3.79 


513.9 


4.27 


11.8 


6.25 


Belgium 


495.9 


5.24 


498.0 


5.61 


-2.1 


6.68 


Canada 


529.1 


1.91 


531.0 


1.74 


-1.8 


1.88 


Czech Republic 


511.9 


3.83 


511.4 


3.20 


0.5 


5.07 


Denmark 


487.6 


3.91 


475.9 


3.54 


11.7 


4.76 


Finland 


534.5 


3.51 


541.0 


2.70 


-6.5 


3.83 


France 


504.1 


4.24 


498.1 


3.77 


6.0 


4.80 


Germany 


489.2 


3.38 


486.7 


3.43 


2.5 


4.73 


Greece 


457.0 


6.08 


464.4 


5.16 


-7.4 


5.75 


Hungary 


495.7 


5.79 


497.3 


5.02 


-1.6 


6.90 


Iceland 


494.8 


3.44 


499.5 


3.01 


-4.7 


4.73 


Ireland 


510.7 


4.23 


516.9 


4.17 


-6.5 


5.52 


Italy 


473.6 


5.62 


482.6 


3.90 


-9.0 


7.73 


Japan 


546.7 


7.18 


554.1 


5.89 


-7.4 


7.20 


Korea, Republic of 


560.7 


4.34 


541.3 


5.13 


19.4 


7.64 


Luxembourg 


441.0 


3.58 


447.5 


3.25 


-6.5 


4.99 


Mexico 


423.3 


4.20 


419.0 


3.85 


4.3 


4.82 


Netherlands 


— 


— 


— 


— 


— 


— 


New Zealand 


522.9 


4.62 


534.8 


3.80 


-11.9 


7.00 


Norway 


498.8 


4.07 


505.4 


3.25 


-6.6 


4.95 


Poland 


486.1 


6.10 


480.0 


6.50 


6.1 


7.37 


Portugal 


456.2 


4.81 


462.5 


4.24 


-6.3 


4.35 


Slovak Republic 


— 


— 


— 


— 


— 


— 


Spain 


492.1 


3.49 


491.4 


3.58 


0.7 


4.01 


Sweden 


512.2 


3.49 


512.6 


2.87 


-0.4 


3.92 


Switzerland 


499.7 


5.69 


492.7 


4.65 


7.0 


5.38 


Turkey 


— 


— 


— 


— 


— 


— 


United States 


497.0 


8.93 


501.8 


6.49 


-4.9 


5.31 


Non-OECD countries 


Hong Kong-China 


— 


— 


— 


— 


— 


— 


Indonesia 


— 


— 


— 


— 


— 


— 


Latvia 


449.3 


6.40 


472.0 


5.83 


-22.7 


5.36 


Liechtenstein 


484.1 


10.93 


468.4 


9.29 


15.7 


14.65 


Macao-China 


— 


— 


— 


— 


— 


— 


Russian Federation 


453.2 


5.36 


467.4 


5.16 


-14.2 


4.50 


Serbia and Montenegro 


— 


— 


— 


— 


— 


— 


Thailand 


— 


— 


— 


— 


— 


— 


Tunisia 


— 


— 


— 


— 


— 


— 


Uruguay 


— 


— 


— 


— 


— 


— 


United Kingdom 1 


535.0 


3.44 


531.4 


3.98 


3.6 


5.20 



See notes at end of table. 
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Table B-22. Average combined reading literacy and science literacy scores of 
15-year-old students, by sex and country: 2000 and 2003 — Continued 









Science literacy in 2003 






Country 


Male 




Female 




Male- 
female 
score point 
difference 






Average 


s.e. 


Average 


s.e. 


s.e. 


OECD average 


502.5 


0.74 


496.8 


0.75 


5.8 


0.90 


OECD countries 


Australia 


524.9 


2.91 


525.2 


2.78 


-0.2 


3.84 


Austria 


489.7 


4.28 


492.3 


4.18 


-2.6 


4.95 


Belgium 


508.8 


3.57 


508.9 


3.48 


-0.1 


5.03 


Canada 


527.4 


2.30 


516.3 


2.22 


11.0 


2.58 


Czech Republic 


526.1 


4.25 


520.4 


4.07 


5.7 


4.90 


Denmark 


484.0 


3.59 


466.8 


3.16 


17.2 


3.18 


Finland 


545.2 


2.55 


551.2 


2.17 


-6.1 


2.77 


France 


511.2 


4.14 


511.3 


3.54 


-0.1 


4.82 


Germany 


505.7 


4.47 


500.0 


4.19 


5.7 


4.77 


Greece 


487.4 


4.78 


475.1 


3.93 


12.3 


4.20 


Hungary 


502.7 


3.31 


504.0 


3.35 


-1.3 


3.68 


Iceland 


489.8 


2.37 


500.0 


2.42 


-10.2 


3.79 


Ireland 


506.4 


3.08 


504.4 


3.88 


2.0 


4.48 


Italy 


489.5 


5.16 


483.6 


3.63 


5.9 


6.31 


Japan 


550.0 


6.00 


545.6 


4.10 


4.4 


6.01 


Korea, Republic of 


545.9 


4.73 


527.5 


5.51 


18.4 


7.05 


Luxembourg 


489.2 


2.53 


476.5 


1.88 


12.7 


3.31 


Mexico 


409.7 


3.87 


400.4 


4.23 


9.3 


4.06 


Netherlands 


526.9 


4.16 


521.8 


3.65 


5.1 


4.66 


New Zealand 


528.8 


2.96 


513.0 


3.36 


15.8 


4.21 


Norway 


485.0 


3.45 


483.3 


3.33 


1.7 


3.61 


Poland 


501.2 


3.24 


494.4 


3.37 


6.8 


3.30 


Portugal 


471.0 


4.01 


464.8 


3.64 


6.3 


3.18 


Slovak Republic 


502.1 


4.28 


487.3 


3.92 


14.8 


3.67 


Spain 


489.1 


3.85 


485.1 


2.59 


4.0 


3.92 


Sweden 


508.6 


3.07 


503.7 


3.46 


4.9 


3.61 


Switzerland 


518.0 


4.99 


507.6 


3.94 


10.3 


5.01 


Turkey 


434.4 


6.66 


434.0 


6.42 


0.5 


5.79 


United States 


493.7 


3.45 


488.7 


3.52 


5.0 


3.27 


Non-OECD countries 


Hong Kong-China 


537.8 


6.05 


541.2 


4.19 


-3.4 


6.00 


Indonesia 


395.6 


3.07 


394.5 


3.84 


1.1 


2.68 


Latvia 


487.2 


5.11 


490.9 


3.90 


-3.8 


4.68 


Liechtenstein 


537.8 


7.72 


511.9 


7.32 


26.0 


12.48 


Macao-China 


529.0 


5.02 


520.6 


4.04 


8.3 


6.78 


Russian Federation 


493.9 


5.35 


484.7 


3.97 


9.2 


4.27 


Serbia and Montenegro 


433.7 


3.69 


438.9 


4.22 


-5.2 


3.82 


Thailand 


424.7 


3.73 


432.7 


3.13 


-8.0 


4.24 


Tunisia 


379.7 


2.75 


389.5 


2.99 


-9.9 


2.64 


Uruguay 


440.5 


3.71 


436.3 


3.59 


4.2 


4.42 


United Kingdom 1 


520.2 


3.14 


516.8 


3.98 


3.4 


5.16 



— Not available. 

'Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE:The male-female score point difference is calculated by subtracting the average scores of females 
from the average scores of males. The OECD average is the average of the national averages of the OECD 
member countries with data available. Because PISA is principally an OECD study, the results for non- 
OECD countries are displayed separately from those of the OECD countries and are not included in the 
OECD average, s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2000 and 2003. 
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Table B-23. Mean International Socioeconomic Index (ISEI) score of 15-year-old students, 
by quarters of the ISEI index and country: 2003 





Mean 




Bottom quarter 


Second quarter 


Third quarter 


Top quarter 


Country 


Index 

score 


s.e. 


Index 

score 


s.e. 


Index 

score 


s.e. 


Index 

score 


s.e. 


Index 

score 


s.e. 


OECD average 


48.8 


0.08 


28.2 


0.04 


42.3 


0.08 


53.2 


0.09 


71.2 


0.13 


OECD countries 


Australia 


52.6 


0.30 


31.6 


0.14 


48.0 


0.07 


58.3 


0.11 


72.5 


0.14 


Austria 


47.1 


0.52 


27.3 


0.19 


40.9 


0.11 


51.4 


0.12 


68.7 


0.28 


Belgium 


50.6 


0.38 


29.0 


0.13 


44.5 


0.13 


56.4 


0.13 


72.4 


0.16 


Canada 


52.6 


0.27 


31.7 


0.11 


47.7 


0.08 


58.1 


0.09 


72.9 


0.15 


Czech Republic 


50.1 


0.34 


32.3 


0.18 


45.7 


0.12 


52.5 


0.05 


69.7 


0.23 


Denmark 


49.3 


0.45 


29.4 


0.19 


44.2 


0.11 


53.2 


0.07 


70.3 


0.29 


Finland 


50.2 


0.36 


28.7 


0.12 


43.4 


0.16 


56.4 


0.14 


72.4 


0.18 


France 


48.7 


0.47 


27.6 


0.20 


42.3 


0.15 


53.6 


0.05 


71.2 


0.26 


Germany 


49.3 


0.42 


29.5 


0.17 


42.6 


0.14 


53.7 


0.06 


71.5 


0.25 


Greece 


46.9 


0.72 


26.9 


0.13 


38.8 


0.13 


51.8 


0.07 


70.3 


0.39 


Hungary 


48.6 


0.33 


30.2 


0.18 


42.3 


0.08 


51.6 


0.11 


70.2 


0.20 


Iceland 


53.7 


0.26 


31.5 


0.20 


48.0 


0.13 


61.7 


0.19 


73.7 


0.25 


Ireland 


48.3 


0.49 


28.5 


0.17 


42.2 


0.11 


52.7 


0.08 


70.0 


0.29 


Italy 


46.8 


0.38 


26.9 


0.16 


40.3 


0.11 


50.6 


0.05 


69.5 


0.38 


Japan 


50.0 


0.31 


33.4 


0.17 


43.9 


0.04 


50.6 


0.08 


72.0 


0.25 


Korea, Republic of 


46.3 


0.36 


28.9 


0.20 


43.5 


0.09 


49.4 


0.06 


63.5 


0.43 


Luxembourg 


48.2 


0.22 


27.3 


0.15 


42.1 


0.13 


52.8 


0.06 


70.5 


0.24 


Mexico 


40.1 


0.68 


22.2 


0.12 


28.9 


0.04 


42.1 


0.28 


67.3 


0.25 


Netherlands 


51.3 


0.38 


30.9 


0.26 


45.4 


0.15 


56.9 


0.20 


71.8 


0.25 


New Zealand 1 


51.5 


0.36 


30.1 


0.19 


46.2 


0.12 


56.8 


0.17 


72.7 


0.26 


Norway 


54.6 


0.39 


35.0 


0.20 


49.0 


0.12 


60.6 


0.16 


73.9 


0.21 


Poland 


45.0 


0.34 


26.9 


0.21 


39.5 


0.11 


49.1 


0.10 


64.4 


0.34 


Portugal 


43.1 


0.54 


26.4 


0.14 


33.9 


0.08 


46.6 


0.19 


65.5 


0.53 


Slovak Republic 


48.8 


0.40 


29.3 


0.17 


41.4 


0.09 


53.1 


0.10 


71.5 


0.21 


Spain 


44.3 


0.58 


26.2 


0.13 


35.5 


0.14 


49.3 


0.11 


66.1 


0.39 


Sweden 


50.6 


0.38 


30.4 


0.18 


44.1 


0.14 


56.1 


0.17 


71.9 


0.21 


Switzerland 


49.3 


0.43 


29.4 


0.14 


43.1 


0.14 


53.5 


0.08 


71.1 


0.27 


Turkey 


41.6 


0.75 


23.7 


0.29 


33.6 


0.15 


47.2 


0.10 


61 .8 


0.77 


United States 


54.6 


0.37 


32.6 


0.21 


49.9 


0.15 


61.4 


0.12 


74.3 


0.21 


Non-OECD countries 


Hong Kong-China 


41.1 


0.45 


25.9 


0.14 


34.9 


0.07 


45.1 


0.13 


58.7 


0.37 


Indonesia 


33.6 


0.61 


16.0 


0.00 


24.1 


0.15 


34.6 


0.33 


59.9 


0.42 


Latvia 


50.3 


0.52 


29.1 


0.23 


44.2 


0.16 


54.8 


0.14 


73.0 


0.30 


Liechtenstein 


50.7 


0.75 


30.8 


0.63 


47.4 


0.52 


55.0 


0.09 


70.0 


0.67 


Macao-China 


39.4 


0.40 


25.8 


0.32 


34.4 


0.12 


41.7 


0.25 


55.9 


0.52 


Russian Federation 


49.9 


0.38 


30.8 


0.16 


40.8 


0.10 


54.2 


0.21 


73.6 


0.20 


Serbia and Montenegro 


48.1 


0.53 


28.3 


0.20 


41.2 


0.12 


51.4 


0.11 


71.4 


0.38 


Thailand 


36.0 


0.43 


22.1 


0.14 


26.7 


0.13 


35.6 


0.13 


59.6 


0.41 


Tunisia 


37.5 


0.60 


18.0 


0.17 


29.2 


0.18 


39.6 


0.19 


63.1 


0.44 


Uruguay 


46.1 


0.48 


25.2 


0.16 


37.8 


0.15 


50.8 


0.12 


70.8 


0.36 


United Kingdom 2 


49.6 


0.39 


28.5 


0.14 


43.0 


0.14 


53.2 


0.09 


71.6 


0.19 



'The item response rate for ISEI for New Zealand is below 85 percent. Missing data have not been explicitly accounted for. See 
also table A-2. 

2 Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the national averages of the OECD member countries with data available. Because 
PISA is principally an OECD study, the results for non-OECD countries are displayed separately from those of the OECD 
countries and are not included in the OECD average.The International Socioeconomic Index (ISEI) is an internationally compa- 
rable index of occupational status, with a range of approximately 16 to 90, developed by Ganzeboom, De Graaf, andTreiman 
(1992). s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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Table B-24. Average combined mathematics literacy scores of 15-year-old students, by 
quarters of the International Socioeconomic Index (ISEI) and country: 2003 



Mathematics literacy 

Country Bottom quarter Second quarter Third quarter Top quarter 





Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


OECD average 


455.5 


0.92 


493.2 


0.75 


516.1 


0.73 


547.7 


0.84 


OECD countries 


Australia 


488.5 


2.81 


520.2 


2.74 


539.3 


2.68 


565.6 


2.87 


Austria 


466.7 


4.40 


492.3 


3.65 


523.9 


3.26 


547.7 


4.44 


Belgium 


482.1 


3.69 


527.3 


3.16 


555.5 


2.82 


589.8 


3.26 


Canada 


506.1 


1.96 


530.9 


2.31 


544.1 


2.11 


568.7 


2.79 


Czech Republic 


485.6 


4.01 


507.7 


3.85 


530.0 


3.89 


569.7 


4.25 


Denmark 


480.9 


3.42 


504.0 


3.64 


525.5 


3.94 


554.2 


3.53 


Finland 


515.1 


2.66 


536.3 


2.73 


551.8 


2.92 


576.5 


2.89 


France 


469.4 


3.72 


507.2 


4.22 


524.8 


2.97 


556.6 


3.76 


Germany 


462.7 


4.93 


505.1 


3.32 


528.1 


3.83 


564.6 


3.98 


Greece 


409.3 


4.28 


435.5 


3.75 


450.4 


4.46 


492.9 


5.00 


Hungary 


449.5 


3.93 


472.8 


3.87 


503.0 


3.36 


547.5 


3.92 


Iceland 


497.1 


3.11 


512.4 


3.19 


518.6 


3.06 


537.7 


3.11 


Ireland 


470.8 


3.93 


495.9 


3.22 


512.9 


3.05 


540.9 


3.50 


Italy 


430.5 


4.19 


456.8 


3.93 


478.2 


3.62 


502.2 


4.11 


Japan 


505.1 


5.09 


534.4 


4.75 


543.0 


4.40 


567.6 


6.36 


Korea, Republic of 


511.3 


4.44 


547.4 


3.67 


549.0 


3.58 


567.7 


6.09 


Luxembourg 


448.4 


2.96 


481.1 


2.96 


508.8 


2.55 


542.4 


3.14 


Mexico 


357.4 


4.81 


373.6 


3.87 


394.3 


3.66 


424.1 


4.87 


Netherlands 


501.5 


4.33 


534.8 


3.79 


558.6 


3.48 


584.2 


3.90 


New Zealand 1 


484.8 


3.80 


514.5 


3.38 


532.4 


3.30 


564.3 


3.42 


Norway 


460.8 


3.49 


488.5 


3.64 


506.7 


3.45 


532.6 


3.52 


Poland 


455.0 


3.94 


479.1 


3.22 


497.7 


3.33 


534.4 


3.14 


Portugal 


431.3 


5.28 


447.1 


3.38 


481.0 


3.82 


511.5 


3.83 


Slovak Republic 


456.5 


4.17 


484.2 


3.35 


523.0 


3.54 


544.3 


3.84 


Spain 


454.0 


3.62 


475.1 


2.79 


495.9 


3.18 


519.4 


3.32 


Sweden 


477.1 


3.69 


501.3 


3.13 


517.8 


3.88 


550.7 


4.15 


Switzerland 


487.1 


4.05 


523.9 


4.12 


537.8 


4.86 


568.3 


3.89 


Turkey 


395.3 


5.65 


410.9 


6.74 


419.6 


7.48 


478.8 


12.51 


United States 


447.7 


3.16 


476.9 


3.76 


496.7 


4.03 


529.7 


3.68 


Non-OECD countries 


Hong Kong-China 


532.1 


5.50 


547.4 


5.11 


562.0 


4.15 


574.7 


5.57 


Indonesia 


335.5 


4.30 


355.6 


4.14 


360.9 


4.48 


397.1 


6.28 


Latvia 


456.6 


3.80 


474.7 


4.31 


493.5 


4.55 


514.0 


5.03 


Liechtenstein 


482.5 


10.29 


530.3 


11.22 


552.9 


9.56 


587.5 


10.97 


Macao-China 


521.8 


5.22 


523.3 


6.33 


527.6 


7.45 


540.5 


7.32 


Russian Federation 


443.1 


4.48 


459.3 


5.27 


472.9 


4.89 


501.3 


4.75 


Serbia and Montenegro 


406.0 


3.72 


426.1 


3.81 


448.6 


4.29 


474.8 


4.99 


Thailand 


396.0 


3.60 


398.9 


3.45 


426.5 


4.04 


456.6 


5.17 


Tunisia 


330.8 


3.03 


341.9 


3.97 


360.8 


3.84 


405.6 


6.13 


Uruguay 


388.2 


4.77 


415.1 


3.98 


429.8 


4.17 


477.6 


3.83 


United Kingdom 2 


469.1 


2.92 


500.0 


3.06 


519.1 


3.53 


555.1 


3.43 



’The item response rate for ISEI for New Zealand is below 85 percent. Missing data have not been explicitly accounted for. See 
also table A-2. 

2 Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the national averages of the OECD member countries with data available. Because 
PISA is principally an OECD study, the results for non-OECD countries are displayed separately from those of the OECD 
countries and are not included in the OECD average.The International Socioeconomic Index (ISEI) is an internationally compa- 
rable index of occupational status developed by Ganzeboom, De Graaf, andTreiman (1992). s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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Table B-25. Change in the combined mathematics literacy and problem-solving 
scores per one standard deviation change in the International 
Socioeconomic Index (ISEI) score, by country: 2003 



Country 


Mathematics literacy 


Problem solving 






Change 


s.e. 


Change 


s.e. 


OECD average 


33.7 


0.40 


— 


— 


OECD countries 


Australia 


30.1 


1.35 


27.6 


1.36 


Austria 


30.7 


1.92 


30.2 


1.98 


Belgium 


39.8 


1.71 


37.0 


1.73 


Canada 


24.4 


1.17 


21.8 


1.19 


Czech Republic 


37.5 


1.97 


33.8 


2.12 


Denmark 


28.9 


1.71 


25.4 


1.73 


Finland 


21.7 


1.29 


20.0 


1.48 


France 


31.6 


1.93 


32.9 


2.22 


Germany 


38.0 


1.95 


36.5 


1.85 


Greece 


29.4 


2.11 


31.1 


2.10 


Hungary 


40.8 


2.17 


41.7 


2.34 


Iceland 


14.4 


1.51 


14.3 


1.53 


Ireland 


27.4 


1.89 


28.1 


1.85 


Italy 


27.1 


1.88 


28.4 


2.08 


Japan 


23.0 


3.12 


21.0 


3.01 


Korea, Republic of 


26.4 


3.28 


23.0 


3.06 


Luxembourg 


33.7 


1.56 


33.3 


1.75 


Mexico 


23.5 


1.88 


28.9 


2.40 


Netherlands 


32.3 


2.03 


32.9 


2.00 


New Zealand 1 


29.4 


1.65 


30.6 


1.88 


Norway 


29.2 


1.62 


31.2 


1.95 


Poland 


35.2 


1.82 


33.8 


1.93 


Portugal 


34.3 


1.70 


35.4 


2.20 


Slovak Republic 


33.2 


1.83 


34.4 


2.08 


Spain 


25.4 


1.43 


28.7 


1.58 


Sweden 


28.7 


1.79 


26.9 


1.73 


Switzerland 


30.3 


1.71 


29.4 


1.75 


Turkey 


38.1 


5.87 


34.9 


5.33 


United States 


30.2 


1.37 


31.1 


1.58 


Non-OECD countries 


Hong Kong-China 


22.6 


2.64 


24.9 


2.60 


Indonesia 


22.0 


2.35 


22.2 


2.02 


Latvia 


21.0 


1.69 


19.8 


2.17 


Liechtenstein 


41.2 


5.92 


40.3 


5.44 


Macao-China 


11.7 


3.97 


10.6 


4.33 


Russian Federation 


21.4 


1.77 


24.0 


2.07 


Serbia and Montenegro 


26.0 


1.86 


24.9 


1.75 


Thailand 


26.6 


2.35 


26.0 


2.32 


Tunisia 


28.3 


2.56 


27.2 


2.08 


Uruguay 


31.4 


1.83 


35.6 


2.37 


United Kingdom 2 


31.8 


1.46 


29.7 


1.63 



— Not available. 

’The item response rate for ISEI for New Zealand is below 85 percent. Missing data have not been explicitly 
accounted for. See also table A-2. 

2 Due to low response rates, 2003 data for the United Kingdom are not discussed in this report. 

NOTE:The OECD average is the average of the national averages of the OECD member countries with data 
available. Because PISA is principally an OECD study, the results for non-OECD countries are displayed 
separately from those of the OECD countries and are not included in the OECD average. The International 
Socioeconomic Index (ISEI) is an internationally comparable index of occupational status, with a range of 
approximately 16 to 90, developed by Ganzeboom, De Graaf, andTreiman (1992). The overall linkage of ISEI 
to mathematics literacy and problem solving is examined using the specific change in score on the com- 
bined mathematics literacy scale or problem solving in response to a one standard deviation change in the 
ISEI index score for each country. A greater increase in achievement score in a country implies a stronger 
relationship between socioeconomic status and performance in that country, s.e. means standard error. 
SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International 
Student Assessment (PISA), 2003. 



PISA 2003 Results From the U.S. Perspective 



Table B-26. Average combined mathematics literacy, problem-solving, reading literacy, and 
science literacy scores of U.S. 15-year-old students, by race/ethnicity: 2003 



Race/ethnicity 


Mathematics 

literacy 


Problem 

solving 


Reading 

literacy 


Science 

literacy 


Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


Average 


s.e. 


Total 


482.9 


2.95 


477.4 


3.13 


495.2 


3.22 


491.3 


3.08 


White 


511.6 


2.51 


505.7 


2.54 


524.8 


2.57 


521.6 


2.60 


Black 


417.3 


5.08 


413.2 


5.69 


429.9 


5.62 


422.7 


4.69 


Hispanic 


442.7 


5.13 


435.6 


5.54 


452.6 


5.86 


448.1 


5.63 


Asian 


506.3 


9.79 


505.3 


9.94 


513.1 


9.22 


508.9 


10.59 


More than one race 


502.2 


6.36 


497.5 


7.05 


515.2 


7.35 


517.0 


7.21 


OECD average 


500.0 


0.63 


500.0 


0.65 


494.2 


0.65 


499.6 


0.60 



NOTE: Reporting standards not met for American Indian/Alaska Native and Native Hawaiian/Other Pacific Islander; thus, they are 
included in the total, but not reported separately. Black includes African American and Hispanic includes Latino. Racial categories 
exclude Hispanic origin, s.e. means standard error. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment 
(PISA), 2003. 
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Appendix D: 

PISA Online Resources 
and Publications 
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Online Resources 

The PISA NCES website 
( http://nces.ed.gov/survevs/pisa) provides 
background information on the PISA sur- 
veys, copies of NCES publications that 
relate to PISA, and sample PISA items from 
previous assessments. 

NCES Publications 

The following publications are intended to 
serve as examples of some of the numerous 
reports that have been produced in relation 
to PISA by NCES. All of the publications 
listed here are available at 
ht tp://nces.ed.gov/survevs/pisa . 

Summary Reports 

Lemke, M., Calsyn, C., Lippman, L., Jocelyn, 
L., Kastberg, D., L i u , Y. , Roey, S., Williams, T., 
Kruger, T., and Bairu, G. (2001). Highlights 
from the 2000 Program for International 
Student Assessment (NCES 2002-116). U.S. 
Department of Education, National Center 
for Education Statistics. Washington, DC: 
U.S. Government Printing Office. 

Lemke, M., Calsyn, C., Lippman, L., Jocelyn, 
L., Kastberg, D., Liu,Y.Y., Roey, S., Williams, 

T. , Kruger, T., Bairu, G. (2001). Outcomes of 
Learning: Results from the 2000 Program for 
International Student Assessment of 15- Year- 
Olds in Reading, Mathematics, and Science 
Literacy (NCES 2002-115). U.S. Department 
of Education, National Center for Education 
Statistics. Washington, DC: U.S. 

Government Printing Office. 

Thematic Reports 

Lemke, M., Sen, A., Pahlke, E., Williams, T., 
Kastberg, D., and Jocelyn, L. (forthcoming). 
Characteristics of U.S. 15- Year-Old Low 
Achievers in an International Context: 

Findings from PISA 2000 (NCES 2002-005). 

U. S. Department of Education, National 
Center for Education Statistics. Washington, 
DC: U.S. Government Printing Office. 

Data Products 

U.S. Department of Education, National 
Center for Education Statistics. (2004). 
Program for International Student 
Assessment (PISA) 2000 Data File (NCES 
2004-006). Washington, DC: Author. 



International Publications 

The following publications are intended to 
serve as examples of some of the numerous 
reports that have been produced in relation 
to PISA by the OECD and other internation- 
al organizations. All of the publications list- 
ed here are available at 
ht tp://www.pisa.oecd.org . 

Summary and Achievement Reports 

Organization for Economic Cooperation and 
Development (OECD). (2001). Knowledge and 
Skills for Life: First Results from the OECD 
Programme for International Student 
Assessment. Paris: Author. 

Thematic Reports 

Artelt, C., Baumert, J., Julius-McElvany, N. 
and Peschar, J. (2003) Learners for Life: 
Student Approaches to Learning. Results from 
PISA 2000. Paris: OECD. 

Kirsch, I., de Jong, J., Lafontaine, D., 
McQueen, J., Mendelovits, J., and Monseur, 
C. (2002). Reading for Change: Performance 
and Engagement Across Countries. Results 
from PISA 2000. Paris: OECD. 

Will ms, J.D. (2003). Student Engagement in 
School: A Sense of Belonging and 
Participation. Results from PISA 2000. Paris: 
OECD. 

Technical Reports and Frameworks 

Adams, R (Ed.) (2003). PISA 2000Technical 
Report. Paris: Organization for Economic 
Cooperation and Development (OECD). 

Organization for Economic Cooperation and 
Development (OECD). (2000). Measuring 
Student Knowledge and Skills: The PISA 2000 
Assessment of Reading, Mathematical and 
Scientific Literacy. Paris: Author. 

Organization for Economic Cooperation and 
Development (OECD). (1999). Measuring 
Student Knowledge and Skills: A New 
Framework for Assessment. Paris: Author. 

Organization for Economic Cooperation and 
Development (OECD). (2002). Programme for 
International Student Assessment (PISA): 
Manual for the PISA 2000 Database. Paris: 
Author. 
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